2023/03/23 - AWS Batch - 4 updated api methods
Changes This feature allows Batch to support configuration of ephemeral storage size for jobs running on FARGATE
{'jobDefinitions': {'containerProperties': {'ephemeralStorage': {'sizeInGiB': 'integer'}},
'eksProperties': {'podProperties': {'metadata': {'labels': {'string': 'string'}}}},
'nodeProperties': {'nodeRangeProperties': {'container': {'ephemeralStorage': {'sizeInGiB': 'integer'}}}}}}
Describes a list of job definitions. You can specify a status (such as ACTIVE ) to only return job definitions that match that status.
See also: AWS API Documentation
Request Syntax
client.describe_job_definitions(
jobDefinitions=[
'string',
],
maxResults=123,
jobDefinitionName='string',
status='string',
nextToken='string'
)
list
A list of up to 100 job definitions. Each entry in the list can either be an ARN in the format arn:aws:batch:${Region}:${Account}:job-definition/${JobDefinitionName}:${Revision} or a short version using the form ${JobDefinitionName}:${Revision} .
(string) --
integer
The maximum number of results returned by DescribeJobDefinitions in paginated output. When this parameter is used, DescribeJobDefinitions only returns maxResults results in a single page and a nextToken response element. The remaining results of the initial request can be seen by sending another DescribeJobDefinitions request with the returned nextToken value. This value can be between 1 and 100. If this parameter isn't used, then DescribeJobDefinitions returns up to 100 results and a nextToken value if applicable.
string
The name of the job definition to describe.
string
The status used to filter job definitions.
string
The nextToken value returned from a previous paginated DescribeJobDefinitions request where maxResults was used and the results exceeded the value of that parameter. Pagination continues from the end of the previous results that returned the nextToken value. This value is null when there are no more results to return.
Note
Treat this token as an opaque identifier that's only used to retrieve the next items in a list and not for other programmatic purposes.
dict
Response Syntax
{
'jobDefinitions': [
{
'jobDefinitionName': 'string',
'jobDefinitionArn': 'string',
'revision': 123,
'status': 'string',
'type': 'string',
'schedulingPriority': 123,
'parameters': {
'string': 'string'
},
'retryStrategy': {
'attempts': 123,
'evaluateOnExit': [
{
'onStatusReason': 'string',
'onReason': 'string',
'onExitCode': 'string',
'action': 'RETRY'|'EXIT'
},
]
},
'containerProperties': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
},
'ephemeralStorage': {
'sizeInGiB': 123
}
},
'timeout': {
'attemptDurationSeconds': 123
},
'nodeProperties': {
'numNodes': 123,
'mainNode': 123,
'nodeRangeProperties': [
{
'targetNodes': 'string',
'container': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
},
'ephemeralStorage': {
'sizeInGiB': 123
}
}
},
]
},
'tags': {
'string': 'string'
},
'propagateTags': True|False,
'platformCapabilities': [
'EC2'|'FARGATE',
],
'eksProperties': {
'podProperties': {
'serviceAccountName': 'string',
'hostNetwork': True|False,
'dnsPolicy': 'string',
'containers': [
{
'name': 'string',
'image': 'string',
'imagePullPolicy': 'string',
'command': [
'string',
],
'args': [
'string',
],
'env': [
{
'name': 'string',
'value': 'string'
},
],
'resources': {
'limits': {
'string': 'string'
},
'requests': {
'string': 'string'
}
},
'volumeMounts': [
{
'name': 'string',
'mountPath': 'string',
'readOnly': True|False
},
],
'securityContext': {
'runAsUser': 123,
'runAsGroup': 123,
'privileged': True|False,
'readOnlyRootFilesystem': True|False,
'runAsNonRoot': True|False
}
},
],
'volumes': [
{
'name': 'string',
'hostPath': {
'path': 'string'
},
'emptyDir': {
'medium': 'string',
'sizeLimit': 'string'
},
'secret': {
'secretName': 'string',
'optional': True|False
}
},
],
'metadata': {
'labels': {
'string': 'string'
}
}
}
},
'containerOrchestrationType': 'ECS'|'EKS'
},
],
'nextToken': 'string'
}
Response Structure
(dict) --
jobDefinitions (list) --
The list of job definitions.
(dict) --
An object that represents an Batch job definition.
jobDefinitionName (string) --
The name of the job definition.
jobDefinitionArn (string) --
The Amazon Resource Name (ARN) for the job definition.
revision (integer) --
The revision of the job definition.
status (string) --
The status of the job definition.
type (string) --
The type of job definition. It's either container or multinode . If the job is run on Fargate resources, then multinode isn't supported. For more information about multi-node parallel jobs, see Creating a multi-node parallel job definition in the Batch User Guide .
schedulingPriority (integer) --
The scheduling priority of the job definition. This only affects jobs in job queues with a fair share policy. Jobs with a higher scheduling priority are scheduled before jobs with a lower scheduling priority.
parameters (dict) --
Default parameters or parameter substitution placeholders that are set in the job definition. Parameters are specified as a key-value pair mapping. Parameters in a SubmitJob request override any corresponding parameter defaults from the job definition. For more information about specifying parameters, see Job definition parameters in the Batch User Guide .
(string) --
(string) --
retryStrategy (dict) --
The retry strategy to use for failed jobs that are submitted with this job definition.
attempts (integer) --
The number of times to move a job to the RUNNABLE status. You can specify between 1 and 10 attempts. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.
evaluateOnExit (list) --
Array of up to 5 objects that specify the conditions where jobs are retried or failed. If this parameter is specified, then the attempts parameter must also be specified. If none of the listed conditions match, then the job is retried.
(dict) --
Specifies an array of up to 5 conditions to be met, and an action to take (RETRY or EXIT ) if all conditions are met. If none of the EvaluateOnExit conditions in a RetryStrategy match, then the job is retried.
onStatusReason (string) --
Contains a glob pattern to match against the StatusReason returned for a job. The pattern can contain up to 512 characters. It can contain letters, numbers, periods (.), colons (:), and white spaces (including spaces or tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
onReason (string) --
Contains a glob pattern to match against the Reason returned for a job. The pattern can contain up to 512 characters. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces and tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
onExitCode (string) --
Contains a glob pattern to match against the decimal representation of the ExitCode returned for a job. The pattern can be up to 512 characters long. It can contain only numbers, and can end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can contain up to 512 characters.
action (string) --
Specifies the action to take if all of the specified conditions (onStatusReason , onReason , and onExitCode ) are met. The values aren't case sensitive.
containerProperties (dict) --
An object with various properties specific to Amazon ECS based jobs. Valid values are containerProperties , eksProperties , and nodeProperties . Only one can be specified.
image (string) --
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . It can be 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), underscores (_), colons (:), periods (.), forward slashes (/), and number signs (#). This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
Images in Amazon ECR Public repositories use the full registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .
Images in Amazon ECR repositories use the full registry and repository URI (for example, 123456789012.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).
Images in official repositories on Docker Hub use a single name (for example, ubuntu or mongo ).
Images in other repositories on Docker Hub are qualified with an organization name (for example, amazon/amazon-ecs-agent ).
Images in other online repositories are qualified further by a domain name (for example, quay.io/assemblyline/ubuntu ).
vcpus (integer) --
This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
memory (integer) --
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs that run on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
command (list) --
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
(string) --
jobRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
executionRoleArn (string) --
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
volumes (list) --
A list of data volumes used in a job.
(dict) --
A data volume that's used in a job's container properties.
host (dict) --
The contents of the host parameter determine whether your data volume persists on the host container instance and where it's stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers that are associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
sourcePath (string) --
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources. Don't provide this for these jobs.
name (string) --
The name of the volume. It can be up to 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
efsVolumeConfiguration (dict) --
This parameter is specified when you're using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
fileSystemId (string) --
The Amazon EFS file system ID to use.
rootDirectory (string) --
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
transitEncryption (string) --
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
transitEncryptionPort (integer) --
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
authorizationConfig (dict) --
The authorization configuration details for the Amazon EFS file system.
accessPointId (string) --
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which enforces the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
iam (string) --
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
environment (list) --
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
mountPoints (list) --
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
(dict) --
Details for a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
containerPath (string) --
The path on the container where the host volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
sourceVolume (string) --
The name of the volume to mount.
readonlyRootFilesystem (boolean) --
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
privileged (boolean) --
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
ulimits (list) --
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
hardLimit (integer) --
The hard limit for the ulimit type.
name (string) --
The type of the ulimit .
softLimit (integer) --
The soft limit for the ulimit type.
user (string) --
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
instanceType (string) --
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
resourceRequirements (list) --
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) --
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. Make sure that the number of GPUs reserved for all containers in a job doesn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs aren't available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU = 0.25
value = 1024
VCPU = 0.25 or 0.5
value = 2048
VCPU = 0.25, 0.5, or 1
value = 3072
VCPU = 0.5, or 1
value = 4096
VCPU = 0.5, 1, or 2
value = 5120, 6144, or 7168
VCPU = 1 or 2
value = 8192
VCPU = 1, 2, or 4
value = 9216, 10240, 11264, 12288, 13312, 14336, or 15360
VCPU = 2 or 4
value = 16384
VCPU = 2, 4, or 8
value = 17408, 18432, 19456, 21504, 22528, 23552, 25600, 26624, 27648, 29696, or 30720
VCPU = 4
value = 20480, 24576, or 28672
VCPU = 4 or 8
value = 36864, 45056, 53248, or 61440
VCPU = 8
value = 32768, 40960, 49152, or 57344
VCPU = 8 or 16
value = 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
VCPU = 16
type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
The default for the Fargate On-Demand vCPU resource count quota is 6 vCPUs. For more information about Fargate quotas, see Fargate quotas in the Amazon Web Services General Reference .
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, 4, 8, and 16
value = 0.25
MEMORY = 512, 1024, or 2048
value = 0.5
MEMORY = 1024, 2048, 3072, or 4096
value = 1
MEMORY = 2048, 3072, 4096, 5120, 6144, 7168, or 8192
value = 2
MEMORY = 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
value = 4
MEMORY = 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
value = 8
MEMORY = 16384, 20480, 24576, 28672, 32768, 36864, 40960, 45056, 49152, 53248, 57344, or 61440
value = 16
MEMORY = 32768, 40960, 49152, 57344, 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
type (string) --
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
linuxParameters (dict) --
Linux-specific modifications that are applied to the container, such as details for device mappings.
devices (list) --
Any of the host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
(dict) --
An object that represents a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
hostPath (string) --
The path for the device on the host container instance.
containerPath (string) --
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
permissions (list) --
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
(string) --
initProcessEnabled (boolean) --
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
sharedMemorySize (integer) --
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
tmpfs (list) --
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide this parameter for this resource type.
(dict) --
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
containerPath (string) --
The absolute file path in the container where the tmpfs volume is mounted.
size (integer) --
The size (in MiB) of the tmpfs volume.
mountOptions (list) --
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
(string) --
maxSwap (integer) --
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance that it's running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
swappiness (integer) --
You can use this parameter to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping to not occur unless absolutely necessary. A swappiness value of 100 causes pages to be swapped aggressively. Valid values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Swap space must be enabled and allocated on the container instance for the containers to use.
Note
By default, the Amazon ECS optimized AMIs don't have swap enabled. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
The swap space parameters are only supported for job definitions using EC2 resources.
If the maxSwap and swappiness parameters are omitted from a job definition, each container has a default swappiness value of 60. Moreover, the total swap usage is limited to two times the memory reservation of the container.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
logConfiguration (dict) --
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
logDriver (string) --
The log driver to use for the container. The valid values that are listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information including usage and options, see Fluentd logging driver in the Docker documentation .
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information including usage and options, see Graylog Extended Format logging driver in the Docker documentation .
journald
Specifies the journald logging driver. For more information including usage and options, see Journald logging driver in the Docker documentation .
json-file
Specifies the JSON file logging driver. For more information including usage and options, see JSON File logging driver in the Docker documentation .
splunk
Specifies the Splunk logging driver. For more information including usage and options, see Splunk logging driver in the Docker documentation .
syslog
Specifies the syslog logging driver. For more information including usage and options, see Syslog logging driver in the Docker documentation .
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
options (dict) --
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
(string) --
(string) --
secretOptions (list) --
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
secrets (list) --
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
networkConfiguration (dict) --
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
assignPublicIp (string) --
Indicates whether the job has a public IP address. For a job that's running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking in the Amazon Elastic Container Service Developer Guide . The default value is "DISABLED ".
fargatePlatformConfiguration (dict) --
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
platformVersion (string) --
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
ephemeralStorage (dict) --
The amount of ephemeral storage to allocate for the task. This parameter is used to expand the total amount of ephemeral storage available, beyond the default amount, for tasks hosted on Fargate.
sizeInGiB (integer) --
The total amount, in GiB, of ephemeral storage to set for the task. The minimum supported value is 21 GiB and the maximum supported value is 200 GiB.
timeout (dict) --
The timeout time for jobs that are submitted with this job definition. After the amount of time you specify passes, Batch terminates your jobs if they aren't finished.
attemptDurationSeconds (integer) --
The job timeout time (in seconds) that's measured from the job attempt's startedAt timestamp. After this time passes, Batch terminates your jobs if they aren't finished. The minimum value for the timeout is 60 seconds.
For array jobs, the timeout applies to the child jobs, not to the parent array job.
For multi-node parallel (MNP) jobs, the timeout applies to the whole job, not to the individual nodes.
nodeProperties (dict) --
An object with various properties that are specific to multi-node parallel jobs. Valid values are containerProperties , eksProperties , and nodeProperties . Only one can be specified.
Note
If the job runs on Fargate resources, don't specify nodeProperties . Use containerProperties instead.
numNodes (integer) --
The number of nodes that are associated with a multi-node parallel job.
mainNode (integer) --
Specifies the node index for the main node of a multi-node parallel job. This node index value must be fewer than the number of nodes.
nodeRangeProperties (list) --
A list of node ranges and their properties that are associated with a multi-node parallel job.
(dict) --
An object that represents the properties of the node range for a multi-node parallel job.
targetNodes (string) --
The range of nodes, using node index values. A range of 0:3 indicates nodes with index values of 0 through 3 . If the starting range value is omitted (:n ), then 0 is used to start the range. If the ending range value is omitted (n: ), then the highest possible node index is used to end the range. Your accumulative node ranges must account for all nodes (0:n ). You can nest node ranges (for example, 0:10 and 4:5 ). In this case, the 4:5 range properties override the 0:10 properties.
container (dict) --
The container details for the node range.
image (string) --
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . It can be 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), underscores (_), colons (:), periods (.), forward slashes (/), and number signs (#). This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
Images in Amazon ECR Public repositories use the full registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .
Images in Amazon ECR repositories use the full registry and repository URI (for example, 123456789012.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).
Images in official repositories on Docker Hub use a single name (for example, ubuntu or mongo ).
Images in other repositories on Docker Hub are qualified with an organization name (for example, amazon/amazon-ecs-agent ).
Images in other online repositories are qualified further by a domain name (for example, quay.io/assemblyline/ubuntu ).
vcpus (integer) --
This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
memory (integer) --
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs that run on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
command (list) --
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
(string) --
jobRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
executionRoleArn (string) --
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
volumes (list) --
A list of data volumes used in a job.
(dict) --
A data volume that's used in a job's container properties.
host (dict) --
The contents of the host parameter determine whether your data volume persists on the host container instance and where it's stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers that are associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
sourcePath (string) --
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources. Don't provide this for these jobs.
name (string) --
The name of the volume. It can be up to 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
efsVolumeConfiguration (dict) --
This parameter is specified when you're using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
fileSystemId (string) --
The Amazon EFS file system ID to use.
rootDirectory (string) --
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
transitEncryption (string) --
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
transitEncryptionPort (integer) --
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
authorizationConfig (dict) --
The authorization configuration details for the Amazon EFS file system.
accessPointId (string) --
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which enforces the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
iam (string) --
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
environment (list) --
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
mountPoints (list) --
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
(dict) --
Details for a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
containerPath (string) --
The path on the container where the host volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
sourceVolume (string) --
The name of the volume to mount.
readonlyRootFilesystem (boolean) --
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
privileged (boolean) --
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
ulimits (list) --
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
hardLimit (integer) --
The hard limit for the ulimit type.
name (string) --
The type of the ulimit .
softLimit (integer) --
The soft limit for the ulimit type.
user (string) --
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
instanceType (string) --
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
resourceRequirements (list) --
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) --
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. Make sure that the number of GPUs reserved for all containers in a job doesn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs aren't available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU = 0.25
value = 1024
VCPU = 0.25 or 0.5
value = 2048
VCPU = 0.25, 0.5, or 1
value = 3072
VCPU = 0.5, or 1
value = 4096
VCPU = 0.5, 1, or 2
value = 5120, 6144, or 7168
VCPU = 1 or 2
value = 8192
VCPU = 1, 2, or 4
value = 9216, 10240, 11264, 12288, 13312, 14336, or 15360
VCPU = 2 or 4
value = 16384
VCPU = 2, 4, or 8
value = 17408, 18432, 19456, 21504, 22528, 23552, 25600, 26624, 27648, 29696, or 30720
VCPU = 4
value = 20480, 24576, or 28672
VCPU = 4 or 8
value = 36864, 45056, 53248, or 61440
VCPU = 8
value = 32768, 40960, 49152, or 57344
VCPU = 8 or 16
value = 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
VCPU = 16
type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
The default for the Fargate On-Demand vCPU resource count quota is 6 vCPUs. For more information about Fargate quotas, see Fargate quotas in the Amazon Web Services General Reference .
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, 4, 8, and 16
value = 0.25
MEMORY = 512, 1024, or 2048
value = 0.5
MEMORY = 1024, 2048, 3072, or 4096
value = 1
MEMORY = 2048, 3072, 4096, 5120, 6144, 7168, or 8192
value = 2
MEMORY = 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
value = 4
MEMORY = 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
value = 8
MEMORY = 16384, 20480, 24576, 28672, 32768, 36864, 40960, 45056, 49152, 53248, 57344, or 61440
value = 16
MEMORY = 32768, 40960, 49152, 57344, 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
type (string) --
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
linuxParameters (dict) --
Linux-specific modifications that are applied to the container, such as details for device mappings.
devices (list) --
Any of the host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
(dict) --
An object that represents a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
hostPath (string) --
The path for the device on the host container instance.
containerPath (string) --
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
permissions (list) --
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
(string) --
initProcessEnabled (boolean) --
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
sharedMemorySize (integer) --
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
tmpfs (list) --
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide this parameter for this resource type.
(dict) --
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
containerPath (string) --
The absolute file path in the container where the tmpfs volume is mounted.
size (integer) --
The size (in MiB) of the tmpfs volume.
mountOptions (list) --
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
(string) --
maxSwap (integer) --
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance that it's running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
swappiness (integer) --
You can use this parameter to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping to not occur unless absolutely necessary. A swappiness value of 100 causes pages to be swapped aggressively. Valid values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Swap space must be enabled and allocated on the container instance for the containers to use.
Note
By default, the Amazon ECS optimized AMIs don't have swap enabled. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
The swap space parameters are only supported for job definitions using EC2 resources.
If the maxSwap and swappiness parameters are omitted from a job definition, each container has a default swappiness value of 60. Moreover, the total swap usage is limited to two times the memory reservation of the container.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
logConfiguration (dict) --
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
logDriver (string) --
The log driver to use for the container. The valid values that are listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information including usage and options, see Fluentd logging driver in the Docker documentation .
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information including usage and options, see Graylog Extended Format logging driver in the Docker documentation .
journald
Specifies the journald logging driver. For more information including usage and options, see Journald logging driver in the Docker documentation .
json-file
Specifies the JSON file logging driver. For more information including usage and options, see JSON File logging driver in the Docker documentation .
splunk
Specifies the Splunk logging driver. For more information including usage and options, see Splunk logging driver in the Docker documentation .
syslog
Specifies the syslog logging driver. For more information including usage and options, see Syslog logging driver in the Docker documentation .
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
options (dict) --
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
(string) --
(string) --
secretOptions (list) --
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
secrets (list) --
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
networkConfiguration (dict) --
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
assignPublicIp (string) --
Indicates whether the job has a public IP address. For a job that's running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking in the Amazon Elastic Container Service Developer Guide . The default value is "DISABLED ".
fargatePlatformConfiguration (dict) --
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
platformVersion (string) --
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
ephemeralStorage (dict) --
The amount of ephemeral storage to allocate for the task. This parameter is used to expand the total amount of ephemeral storage available, beyond the default amount, for tasks hosted on Fargate.
sizeInGiB (integer) --
The total amount, in GiB, of ephemeral storage to set for the task. The minimum supported value is 21 GiB and the maximum supported value is 200 GiB.
tags (dict) --
The tags that are applied to the job definition.
(string) --
(string) --
propagateTags (boolean) --
Specifies whether to propagate the tags from the job or job definition to the corresponding Amazon ECS task. If no value is specified, the tags aren't propagated. Tags can only be propagated to the tasks when the tasks are created. For tags with the same name, job tags are given priority over job definitions tags. If the total number of combined tags from the job and job definition is over 50, the job is moved to the FAILED state.
platformCapabilities (list) --
The platform capabilities required by the job definition. If no value is specified, it defaults to EC2 . Jobs run on Fargate resources specify FARGATE .
(string) --
eksProperties (dict) --
An object with various properties that are specific to Amazon EKS based jobs. Valid values are containerProperties , eksProperties , and nodeProperties . Only one can be specified.
podProperties (dict) --
The properties for the Kubernetes pod resources of a job.
serviceAccountName (string) --
The name of the service account that's used to run the pod. For more information, see Kubernetes service accounts and Configure a Kubernetes service account to assume an IAM role in the Amazon EKS User Guide and Configure service accounts for pods in the Kubernetes documentation .
hostNetwork (boolean) --
Indicates if the pod uses the hosts' network IP address. The default value is true . Setting this to false enables the Kubernetes pod networking model. Most Batch workloads are egress-only and don't require the overhead of IP allocation for each pod for incoming connections. For more information, see Host namespaces and Pod networking in the Kubernetes documentation .
dnsPolicy (string) --
The DNS policy for the pod. The default value is ClusterFirst . If the hostNetwork parameter is not specified, the default is ClusterFirstWithHostNet . ClusterFirst indicates that any DNS query that does not match the configured cluster domain suffix is forwarded to the upstream nameserver inherited from the node. For more information, see Pod's DNS policy in the Kubernetes documentation .
Valid values: Default | ClusterFirst | ClusterFirstWithHostNet
containers (list) --
The properties of the container that's used on the Amazon EKS pod.
(dict) --
EKS container properties are used in job definitions for Amazon EKS based job definitions to describe the properties for a container node in the pod that's launched as part of a job. This can't be specified for Amazon ECS based job definitions.
name (string) --
The name of the container. If the name isn't specified, the default name "Default " is used. Each container in a pod must have a unique name.
image (string) --
The Docker image used to start the container.
imagePullPolicy (string) --
The image pull policy for the container. Supported values are Always , IfNotPresent , and Never . This parameter defaults to IfNotPresent . However, if the :latest tag is specified, it defaults to Always . For more information, see Updating images in the Kubernetes documentation .
command (list) --
The entrypoint for the container. This isn't run within a shell. If this isn't specified, the ENTRYPOINT of the container image is used. Environment variable references are expanded using the container's environment.
If the referenced environment variable doesn't exist, the reference in the command isn't changed. For example, if the reference is to "$(NAME1) " and the NAME1 environment variable doesn't exist, the command string will remain "$(NAME1) ." $$ is replaced with $ and the resulting string isn't expanded. For example, $$(VAR_NAME) will be passed as $(VAR_NAME) whether or not the VAR_NAME environment variable exists. The entrypoint can't be updated. For more information, see ENTRYPOINT in the Dockerfile reference and Define a command and arguments for a container and Entrypoint in the Kubernetes documentation .
(string) --
args (list) --
An array of arguments to the entrypoint. If this isn't specified, the CMD of the container image is used. This corresponds to the args member in the Entrypoint portion of the Pod in Kubernetes. Environment variable references are expanded using the container's environment.
If the referenced environment variable doesn't exist, the reference in the command isn't changed. For example, if the reference is to "$(NAME1) " and the NAME1 environment variable doesn't exist, the command string will remain "$(NAME1) ." $$ is replaced with $ , and the resulting string isn't expanded. For example, $$(VAR_NAME) is passed as $(VAR_NAME) whether or not the VAR_NAME environment variable exists. For more information, see CMD in the Dockerfile reference and Define a command and arguments for a pod in the Kubernetes documentation .
(string) --
env (list) --
The environment variables to pass to a container.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
An environment variable.
name (string) --
The name of the environment variable.
value (string) --
The value of the environment variable.
resources (dict) --
The type and amount of resources to assign to a container. The supported resources include memory , cpu , and nvidia.com/gpu . For more information, see Resource management for pods and containers in the Kubernetes documentation .
limits (dict) --
The type and quantity of the resources to reserve for the container. The values vary based on the name that's specified. Resources can be requested using either the limits or the requests objects.
memory
The memory hard limit (in MiB) for the container, using whole integers, with a "Mi" suffix. If your container attempts to exceed the memory specified, the container is terminated. You must specify at least 4 MiB of memory for a job. memory can be specified in limits , requests , or both. If memory is specified in both places, then the value that's specified in limits must be equal to the value that's specified in requests .
Note
To maximize your resource utilization, provide your jobs with as much memory as possible for the specific instance type that you are using. To learn how, see Memory management in the Batch User Guide .
cpu
The number of CPUs that's reserved for the container. Values must be an even multiple of 0.25 . cpu can be specified in limits , requests , or both. If cpu is specified in both places, then the value that's specified in limits must be at least as large as the value that's specified in requests .
nvidia.com/gpu
The number of GPUs that's reserved for the container. Values must be a whole integer. memory can be specified in limits , requests , or both. If memory is specified in both places, then the value that's specified in limits must be equal to the value that's specified in requests .
(string) --
(string) --
requests (dict) --
The type and quantity of the resources to request for the container. The values vary based on the name that's specified. Resources can be requested by using either the limits or the requests objects.
memory
The memory hard limit (in MiB) for the container, using whole integers, with a "Mi" suffix. If your container attempts to exceed the memory specified, the container is terminated. You must specify at least 4 MiB of memory for a job. memory can be specified in limits , requests , or both. If memory is specified in both, then the value that's specified in limits must be equal to the value that's specified in requests .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
cpu
The number of CPUs that are reserved for the container. Values must be an even multiple of 0.25 . cpu can be specified in limits , requests , or both. If cpu is specified in both, then the value that's specified in limits must be at least as large as the value that's specified in requests .
nvidia.com/gpu
The number of GPUs that are reserved for the container. Values must be a whole integer. nvidia.com/gpu can be specified in limits , requests , or both. If nvidia.com/gpu is specified in both, then the value that's specified in limits must be equal to the value that's specified in requests .
(string) --
(string) --
volumeMounts (list) --
The volume mounts for the container. Batch supports emptyDir , hostPath , and secret volume types. For more information about volumes and volume mounts in Kubernetes, see Volumes in the Kubernetes documentation .
(dict) --
The volume mounts for a container for an Amazon EKS job. For more information about volumes and volume mounts in Kubernetes, see Volumes in the Kubernetes documentation .
name (string) --
The name the volume mount. This must match the name of one of the volumes in the pod.
mountPath (string) --
The path on the container where the volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
securityContext (dict) --
The security context for a job. For more information, see Configure a security context for a pod or container in the Kubernetes documentation .
runAsUser (integer) --
When this parameter is specified, the container is run as the specified user ID (uid ). If this parameter isn't specified, the default is the user that's specified in the image metadata. This parameter maps to RunAsUser and MustRanAs policy in the Users and groups pod security policies in the Kubernetes documentation .
runAsGroup (integer) --
When this parameter is specified, the container is run as the specified group ID (gid ). If this parameter isn't specified, the default is the group that's specified in the image metadata. This parameter maps to RunAsGroup and MustRunAs policy in the Users and groups pod security policies in the Kubernetes documentation .
privileged (boolean) --
When this parameter is true , the container is given elevated permissions on the host container instance. The level of permissions are similar to the root user permissions. The default value is false . This parameter maps to privileged policy in the Privileged pod security policies in the Kubernetes documentation .
readOnlyRootFilesystem (boolean) --
When this parameter is true , the container is given read-only access to its root file system. The default value is false . This parameter maps to ReadOnlyRootFilesystem policy in the Volumes and file systems pod security policies in the Kubernetes documentation .
runAsNonRoot (boolean) --
When this parameter is specified, the container is run as a user with a uid other than 0. If this parameter isn't specified, so such rule is enforced. This parameter maps to RunAsUser and MustRunAsNonRoot policy in the Users and groups pod security policies in the Kubernetes documentation .
volumes (list) --
Specifies the volumes for a job definition that uses Amazon EKS resources.
(dict) --
Specifies an Amazon EKS volume for a job definition.
name (string) --
The name of the volume. The name must be allowed as a DNS subdomain name. For more information, see DNS subdomain names in the Kubernetes documentation .
hostPath (dict) --
Specifies the configuration of a Kubernetes hostPath volume. For more information, see hostPath in the Kubernetes documentation .
path (string) --
The path of the file or directory on the host to mount into containers on the pod.
emptyDir (dict) --
Specifies the configuration of a Kubernetes emptyDir volume. For more information, see emptyDir in the Kubernetes documentation .
medium (string) --
The medium to store the volume. The default value is an empty string, which uses the storage of the node.
""
(Default) Use the disk storage of the node.
"Memory"
Use the tmpfs volume that's backed by the RAM of the node. Contents of the volume are lost when the node reboots, and any storage on the volume counts against the container's memory limit.
sizeLimit (string) --
The maximum size of the volume. By default, there's no maximum size defined.
secret (dict) --
Specifies the configuration of a Kubernetes secret volume. For more information, see secret in the Kubernetes documentation .
secretName (string) --
The name of the secret. The name must be allowed as a DNS subdomain name. For more information, see DNS subdomain names in the Kubernetes documentation .
optional (boolean) --
Specifies whether the secret or the secret's keys must be defined.
metadata (dict) --
labels (dict) --
(string) --
(string) --
containerOrchestrationType (string) --
The orchestration type of the compute environment. The valid values are ECS (default) or EKS .
nextToken (string) --
The nextToken value to include in a future DescribeJobDefinitions request. When the results of a DescribeJobDefinitions request exceed maxResults , this value can be used to retrieve the next page of results. This value is null when there are no more results to return.
{'jobs': {'container': {'ephemeralStorage': {'sizeInGiB': 'integer'}},
'nodeProperties': {'nodeRangeProperties': {'container': {'ephemeralStorage': {'sizeInGiB': 'integer'}}}}}}
Describes a list of Batch jobs.
See also: AWS API Documentation
Request Syntax
client.describe_jobs(
jobs=[
'string',
]
)
list
[REQUIRED]
A list of up to 100 job IDs.
(string) --
dict
Response Syntax
{
'jobs': [
{
'jobArn': 'string',
'jobName': 'string',
'jobId': 'string',
'jobQueue': 'string',
'status': 'SUBMITTED'|'PENDING'|'RUNNABLE'|'STARTING'|'RUNNING'|'SUCCEEDED'|'FAILED',
'shareIdentifier': 'string',
'schedulingPriority': 123,
'attempts': [
{
'container': {
'containerInstanceArn': 'string',
'taskArn': 'string',
'exitCode': 123,
'reason': 'string',
'logStreamName': 'string',
'networkInterfaces': [
{
'attachmentId': 'string',
'ipv6Address': 'string',
'privateIpv4Address': 'string'
},
]
},
'startedAt': 123,
'stoppedAt': 123,
'statusReason': 'string'
},
],
'statusReason': 'string',
'createdAt': 123,
'retryStrategy': {
'attempts': 123,
'evaluateOnExit': [
{
'onStatusReason': 'string',
'onReason': 'string',
'onExitCode': 'string',
'action': 'RETRY'|'EXIT'
},
]
},
'startedAt': 123,
'stoppedAt': 123,
'dependsOn': [
{
'jobId': 'string',
'type': 'N_TO_N'|'SEQUENTIAL'
},
],
'jobDefinition': 'string',
'parameters': {
'string': 'string'
},
'container': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'privileged': True|False,
'user': 'string',
'exitCode': 123,
'reason': 'string',
'containerInstanceArn': 'string',
'taskArn': 'string',
'logStreamName': 'string',
'instanceType': 'string',
'networkInterfaces': [
{
'attachmentId': 'string',
'ipv6Address': 'string',
'privateIpv4Address': 'string'
},
],
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
},
'ephemeralStorage': {
'sizeInGiB': 123
}
},
'nodeDetails': {
'nodeIndex': 123,
'isMainNode': True|False
},
'nodeProperties': {
'numNodes': 123,
'mainNode': 123,
'nodeRangeProperties': [
{
'targetNodes': 'string',
'container': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
},
'ephemeralStorage': {
'sizeInGiB': 123
}
}
},
]
},
'arrayProperties': {
'statusSummary': {
'string': 123
},
'size': 123,
'index': 123
},
'timeout': {
'attemptDurationSeconds': 123
},
'tags': {
'string': 'string'
},
'propagateTags': True|False,
'platformCapabilities': [
'EC2'|'FARGATE',
],
'eksProperties': {
'podProperties': {
'serviceAccountName': 'string',
'hostNetwork': True|False,
'dnsPolicy': 'string',
'containers': [
{
'name': 'string',
'image': 'string',
'imagePullPolicy': 'string',
'command': [
'string',
],
'args': [
'string',
],
'env': [
{
'name': 'string',
'value': 'string'
},
],
'resources': {
'limits': {
'string': 'string'
},
'requests': {
'string': 'string'
}
},
'exitCode': 123,
'reason': 'string',
'volumeMounts': [
{
'name': 'string',
'mountPath': 'string',
'readOnly': True|False
},
],
'securityContext': {
'runAsUser': 123,
'runAsGroup': 123,
'privileged': True|False,
'readOnlyRootFilesystem': True|False,
'runAsNonRoot': True|False
}
},
],
'volumes': [
{
'name': 'string',
'hostPath': {
'path': 'string'
},
'emptyDir': {
'medium': 'string',
'sizeLimit': 'string'
},
'secret': {
'secretName': 'string',
'optional': True|False
}
},
],
'podName': 'string',
'nodeName': 'string'
}
},
'eksAttempts': [
{
'containers': [
{
'exitCode': 123,
'reason': 'string'
},
],
'podName': 'string',
'nodeName': 'string',
'startedAt': 123,
'stoppedAt': 123,
'statusReason': 'string'
},
],
'isCancelled': True|False,
'isTerminated': True|False
},
]
}
Response Structure
(dict) --
jobs (list) --
The list of jobs.
(dict) --
An object that represents an Batch job.
jobArn (string) --
The Amazon Resource Name (ARN) of the job.
jobName (string) --
The job name.
jobId (string) --
The job ID.
jobQueue (string) --
The Amazon Resource Name (ARN) of the job queue that the job is associated with.
status (string) --
The current status for the job.
Note
If your jobs don't progress to STARTING , see Jobs stuck in RUNNABLE status in the troubleshooting section of the Batch User Guide .
shareIdentifier (string) --
The share identifier for the job.
schedulingPriority (integer) --
The scheduling policy of the job definition. This only affects jobs in job queues with a fair share policy. Jobs with a higher scheduling priority are scheduled before jobs with a lower scheduling priority.
attempts (list) --
A list of job attempts that are associated with this job.
(dict) --
An object that represents a job attempt.
container (dict) --
The details for the container in this job attempt.
containerInstanceArn (string) --
The Amazon Resource Name (ARN) of the Amazon ECS container instance that hosts the job attempt.
taskArn (string) --
The Amazon Resource Name (ARN) of the Amazon ECS task that's associated with the job attempt. Each container attempt receives a task ARN when they reach the STARTING status.
exitCode (integer) --
The exit code for the job attempt. A non-zero exit code is considered failed.
reason (string) --
A short (255 max characters) human-readable string to provide additional details for a running or stopped container.
logStreamName (string) --
The name of the CloudWatch Logs log stream that's associated with the container. The log group for Batch jobs is /aws/batch/job . Each container attempt receives a log stream name when they reach the RUNNING status.
networkInterfaces (list) --
The network interfaces that are associated with the job attempt.
(dict) --
An object that represents the elastic network interface for a multi-node parallel job node.
attachmentId (string) --
The attachment ID for the network interface.
ipv6Address (string) --
The private IPv6 address for the network interface.
privateIpv4Address (string) --
The private IPv4 address for the network interface.
startedAt (integer) --
The Unix timestamp (in milliseconds) for when the attempt was started (when the attempt transitioned from the STARTING state to the RUNNING state).
stoppedAt (integer) --
The Unix timestamp (in milliseconds) for when the attempt was stopped (when the attempt transitioned from the RUNNING state to a terminal state, such as SUCCEEDED or FAILED ).
statusReason (string) --
A short, human-readable string to provide additional details for the current status of the job attempt.
statusReason (string) --
A short, human-readable string to provide more details for the current status of the job.
createdAt (integer) --
The Unix timestamp (in milliseconds) for when the job was created. For non-array jobs and parent array jobs, this is when the job entered the SUBMITTED state. This is specifically at the time SubmitJob was called. For array child jobs, this is when the child job was spawned by its parent and entered the PENDING state.
retryStrategy (dict) --
The retry strategy to use for this job if an attempt fails.
attempts (integer) --
The number of times to move a job to the RUNNABLE status. You can specify between 1 and 10 attempts. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.
evaluateOnExit (list) --
Array of up to 5 objects that specify the conditions where jobs are retried or failed. If this parameter is specified, then the attempts parameter must also be specified. If none of the listed conditions match, then the job is retried.
(dict) --
Specifies an array of up to 5 conditions to be met, and an action to take (RETRY or EXIT ) if all conditions are met. If none of the EvaluateOnExit conditions in a RetryStrategy match, then the job is retried.
onStatusReason (string) --
Contains a glob pattern to match against the StatusReason returned for a job. The pattern can contain up to 512 characters. It can contain letters, numbers, periods (.), colons (:), and white spaces (including spaces or tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
onReason (string) --
Contains a glob pattern to match against the Reason returned for a job. The pattern can contain up to 512 characters. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces and tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
onExitCode (string) --
Contains a glob pattern to match against the decimal representation of the ExitCode returned for a job. The pattern can be up to 512 characters long. It can contain only numbers, and can end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can contain up to 512 characters.
action (string) --
Specifies the action to take if all of the specified conditions (onStatusReason , onReason , and onExitCode ) are met. The values aren't case sensitive.
startedAt (integer) --
The Unix timestamp (in milliseconds) for when the job was started. More specifically, it's when the job transitioned from the STARTING state to the RUNNING state. This parameter isn't provided for child jobs of array jobs or multi-node parallel jobs.
stoppedAt (integer) --
The Unix timestamp (in milliseconds) for when the job was stopped. More specifically, it's when the job transitioned from the RUNNING state to a terminal state, such as SUCCEEDED or FAILED .
dependsOn (list) --
A list of job IDs that this job depends on.
(dict) --
An object that represents an Batch job dependency.
jobId (string) --
The job ID of the Batch job that's associated with this dependency.
type (string) --
The type of the job dependency.
jobDefinition (string) --
The Amazon Resource Name (ARN) of the job definition that this job uses.
parameters (dict) --
Additional parameters that are passed to the job that replace parameter substitution placeholders or override any corresponding parameter defaults from the job definition.
(string) --
(string) --
container (dict) --
An object that represents the details for the container that's associated with the job.
image (string) --
The image used to start the container.
vcpus (integer) --
The number of vCPUs reserved for the container. For jobs that run on EC2 resources, you can specify the vCPU requirement for the job using resourceRequirements , but you can't specify the vCPU requirements in both the vcpus and resourceRequirements object. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. You must specify at least one vCPU. This is required but can be specified in several places. It must be specified for each node at least once.
Note
This parameter isn't applicable to jobs that run on Fargate resources. For jobs that run on Fargate resources, you must specify the vCPU requirement for the job using resourceRequirements .
memory (integer) --
For jobs running on EC2 resources that didn't specify memory requirements using resourceRequirements , the number of MiB of memory reserved for the job. For other jobs, including all run on Fargate resources, see resourceRequirements .
command (list) --
The command that's passed to the container.
(string) --
jobRoleArn (string) --
The Amazon Resource Name (ARN) that's associated with the job when run.
executionRoleArn (string) --
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For more information, see Batch execution IAM role in the Batch User Guide .
volumes (list) --
A list of volumes that are associated with the job.
(dict) --
A data volume that's used in a job's container properties.
host (dict) --
The contents of the host parameter determine whether your data volume persists on the host container instance and where it's stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers that are associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
sourcePath (string) --
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources. Don't provide this for these jobs.
name (string) --
The name of the volume. It can be up to 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
efsVolumeConfiguration (dict) --
This parameter is specified when you're using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
fileSystemId (string) --
The Amazon EFS file system ID to use.
rootDirectory (string) --
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
transitEncryption (string) --
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
transitEncryptionPort (integer) --
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
authorizationConfig (dict) --
The authorization configuration details for the Amazon EFS file system.
accessPointId (string) --
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which enforces the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
iam (string) --
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
environment (list) --
The environment variables to pass to a container.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
mountPoints (list) --
The mount points for data volumes in your container.
(dict) --
Details for a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
containerPath (string) --
The path on the container where the host volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
sourceVolume (string) --
The name of the volume to mount.
readonlyRootFilesystem (boolean) --
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to ` docker run https://docs.docker.com/engine/reference/commandline/run/`__ .
ulimits (list) --
A list of ulimit values to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources.
(dict) --
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
hardLimit (integer) --
The hard limit for the ulimit type.
name (string) --
The type of the ulimit .
softLimit (integer) --
The soft limit for the ulimit type.
privileged (boolean) --
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). The default value is false .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false .
user (string) --
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
exitCode (integer) --
The exit code to return upon completion.
reason (string) --
A short (255 max characters) human-readable string to provide additional details for a running or stopped container.
containerInstanceArn (string) --
The Amazon Resource Name (ARN) of the container instance that the container is running on.
taskArn (string) --
The Amazon Resource Name (ARN) of the Amazon ECS task that's associated with the container job. Each container attempt receives a task ARN when they reach the STARTING status.
logStreamName (string) --
The name of the Amazon CloudWatch Logs log stream that's associated with the container. The log group for Batch jobs is /aws/batch/job . Each container attempt receives a log stream name when they reach the RUNNING status.
instanceType (string) --
The instance type of the underlying host infrastructure of a multi-node parallel job.
Note
This parameter isn't applicable to jobs that are running on Fargate resources.
networkInterfaces (list) --
The network interfaces that are associated with the job.
(dict) --
An object that represents the elastic network interface for a multi-node parallel job node.
attachmentId (string) --
The attachment ID for the network interface.
ipv6Address (string) --
The private IPv6 address for the network interface.
privateIpv4Address (string) --
The private IPv4 address for the network interface.
resourceRequirements (list) --
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) --
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. Make sure that the number of GPUs reserved for all containers in a job doesn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs aren't available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU = 0.25
value = 1024
VCPU = 0.25 or 0.5
value = 2048
VCPU = 0.25, 0.5, or 1
value = 3072
VCPU = 0.5, or 1
value = 4096
VCPU = 0.5, 1, or 2
value = 5120, 6144, or 7168
VCPU = 1 or 2
value = 8192
VCPU = 1, 2, or 4
value = 9216, 10240, 11264, 12288, 13312, 14336, or 15360
VCPU = 2 or 4
value = 16384
VCPU = 2, 4, or 8
value = 17408, 18432, 19456, 21504, 22528, 23552, 25600, 26624, 27648, 29696, or 30720
VCPU = 4
value = 20480, 24576, or 28672
VCPU = 4 or 8
value = 36864, 45056, 53248, or 61440
VCPU = 8
value = 32768, 40960, 49152, or 57344
VCPU = 8 or 16
value = 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
VCPU = 16
type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
The default for the Fargate On-Demand vCPU resource count quota is 6 vCPUs. For more information about Fargate quotas, see Fargate quotas in the Amazon Web Services General Reference .
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, 4, 8, and 16
value = 0.25
MEMORY = 512, 1024, or 2048
value = 0.5
MEMORY = 1024, 2048, 3072, or 4096
value = 1
MEMORY = 2048, 3072, 4096, 5120, 6144, 7168, or 8192
value = 2
MEMORY = 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
value = 4
MEMORY = 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
value = 8
MEMORY = 16384, 20480, 24576, 28672, 32768, 36864, 40960, 45056, 49152, 53248, 57344, or 61440
value = 16
MEMORY = 32768, 40960, 49152, 57344, 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
type (string) --
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
linuxParameters (dict) --
Linux-specific modifications that are applied to the container, such as details for device mappings.
devices (list) --
Any of the host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
(dict) --
An object that represents a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
hostPath (string) --
The path for the device on the host container instance.
containerPath (string) --
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
permissions (list) --
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
(string) --
initProcessEnabled (boolean) --
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
sharedMemorySize (integer) --
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
tmpfs (list) --
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide this parameter for this resource type.
(dict) --
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
containerPath (string) --
The absolute file path in the container where the tmpfs volume is mounted.
size (integer) --
The size (in MiB) of the tmpfs volume.
mountOptions (list) --
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
(string) --
maxSwap (integer) --
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance that it's running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
swappiness (integer) --
You can use this parameter to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping to not occur unless absolutely necessary. A swappiness value of 100 causes pages to be swapped aggressively. Valid values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Swap space must be enabled and allocated on the container instance for the containers to use.
Note
By default, the Amazon ECS optimized AMIs don't have swap enabled. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
The swap space parameters are only supported for job definitions using EC2 resources.
If the maxSwap and swappiness parameters are omitted from a job definition, each container has a default swappiness value of 60. Moreover, the total swap usage is limited to two times the memory reservation of the container.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
logConfiguration (dict) --
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However, the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance. Or, alternatively, it must be configured on a different log server for remote logging options. For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type). Additional log drivers might be available in future releases of the Amazon ECS container agent.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
logDriver (string) --
The log driver to use for the container. The valid values that are listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information including usage and options, see Fluentd logging driver in the Docker documentation .
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information including usage and options, see Graylog Extended Format logging driver in the Docker documentation .
journald
Specifies the journald logging driver. For more information including usage and options, see Journald logging driver in the Docker documentation .
json-file
Specifies the JSON file logging driver. For more information including usage and options, see JSON File logging driver in the Docker documentation .
splunk
Specifies the Splunk logging driver. For more information including usage and options, see Splunk logging driver in the Docker documentation .
syslog
Specifies the syslog logging driver. For more information including usage and options, see Syslog logging driver in the Docker documentation .
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
options (dict) --
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
(string) --
(string) --
secretOptions (list) --
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
secrets (list) --
The secrets to pass to the container. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
networkConfiguration (dict) --
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
assignPublicIp (string) --
Indicates whether the job has a public IP address. For a job that's running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking in the Amazon Elastic Container Service Developer Guide . The default value is "DISABLED ".
fargatePlatformConfiguration (dict) --
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
platformVersion (string) --
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
ephemeralStorage (dict) --
The amount of ephemeral storage to allocate for the task. This parameter is used to expand the total amount of ephemeral storage available, beyond the default amount, for tasks hosted on Fargate.
sizeInGiB (integer) --
The total amount, in GiB, of ephemeral storage to set for the task. The minimum supported value is 21 GiB and the maximum supported value is 200 GiB.
nodeDetails (dict) --
An object that represents the details of a node that's associated with a multi-node parallel job.
nodeIndex (integer) --
The node index for the node. Node index numbering starts at zero. This index is also available on the node with the AWS_BATCH_JOB_NODE_INDEX environment variable.
isMainNode (boolean) --
Specifies whether the current node is the main node for a multi-node parallel job.
nodeProperties (dict) --
An object that represents the node properties of a multi-node parallel job.
Note
This isn't applicable to jobs that are running on Fargate resources.
numNodes (integer) --
The number of nodes that are associated with a multi-node parallel job.
mainNode (integer) --
Specifies the node index for the main node of a multi-node parallel job. This node index value must be fewer than the number of nodes.
nodeRangeProperties (list) --
A list of node ranges and their properties that are associated with a multi-node parallel job.
(dict) --
An object that represents the properties of the node range for a multi-node parallel job.
targetNodes (string) --
The range of nodes, using node index values. A range of 0:3 indicates nodes with index values of 0 through 3 . If the starting range value is omitted (:n ), then 0 is used to start the range. If the ending range value is omitted (n: ), then the highest possible node index is used to end the range. Your accumulative node ranges must account for all nodes (0:n ). You can nest node ranges (for example, 0:10 and 4:5 ). In this case, the 4:5 range properties override the 0:10 properties.
container (dict) --
The container details for the node range.
image (string) --
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . It can be 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), underscores (_), colons (:), periods (.), forward slashes (/), and number signs (#). This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
Images in Amazon ECR Public repositories use the full registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .
Images in Amazon ECR repositories use the full registry and repository URI (for example, 123456789012.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).
Images in official repositories on Docker Hub use a single name (for example, ubuntu or mongo ).
Images in other repositories on Docker Hub are qualified with an organization name (for example, amazon/amazon-ecs-agent ).
Images in other online repositories are qualified further by a domain name (for example, quay.io/assemblyline/ubuntu ).
vcpus (integer) --
This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
memory (integer) --
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs that run on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
command (list) --
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
(string) --
jobRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
executionRoleArn (string) --
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
volumes (list) --
A list of data volumes used in a job.
(dict) --
A data volume that's used in a job's container properties.
host (dict) --
The contents of the host parameter determine whether your data volume persists on the host container instance and where it's stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers that are associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
sourcePath (string) --
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources. Don't provide this for these jobs.
name (string) --
The name of the volume. It can be up to 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
efsVolumeConfiguration (dict) --
This parameter is specified when you're using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
fileSystemId (string) --
The Amazon EFS file system ID to use.
rootDirectory (string) --
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
transitEncryption (string) --
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
transitEncryptionPort (integer) --
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
authorizationConfig (dict) --
The authorization configuration details for the Amazon EFS file system.
accessPointId (string) --
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which enforces the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
iam (string) --
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
environment (list) --
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
mountPoints (list) --
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
(dict) --
Details for a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
containerPath (string) --
The path on the container where the host volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
sourceVolume (string) --
The name of the volume to mount.
readonlyRootFilesystem (boolean) --
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
privileged (boolean) --
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
ulimits (list) --
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
hardLimit (integer) --
The hard limit for the ulimit type.
name (string) --
The type of the ulimit .
softLimit (integer) --
The soft limit for the ulimit type.
user (string) --
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
instanceType (string) --
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
resourceRequirements (list) --
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) --
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. Make sure that the number of GPUs reserved for all containers in a job doesn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs aren't available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU = 0.25
value = 1024
VCPU = 0.25 or 0.5
value = 2048
VCPU = 0.25, 0.5, or 1
value = 3072
VCPU = 0.5, or 1
value = 4096
VCPU = 0.5, 1, or 2
value = 5120, 6144, or 7168
VCPU = 1 or 2
value = 8192
VCPU = 1, 2, or 4
value = 9216, 10240, 11264, 12288, 13312, 14336, or 15360
VCPU = 2 or 4
value = 16384
VCPU = 2, 4, or 8
value = 17408, 18432, 19456, 21504, 22528, 23552, 25600, 26624, 27648, 29696, or 30720
VCPU = 4
value = 20480, 24576, or 28672
VCPU = 4 or 8
value = 36864, 45056, 53248, or 61440
VCPU = 8
value = 32768, 40960, 49152, or 57344
VCPU = 8 or 16
value = 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
VCPU = 16
type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
The default for the Fargate On-Demand vCPU resource count quota is 6 vCPUs. For more information about Fargate quotas, see Fargate quotas in the Amazon Web Services General Reference .
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, 4, 8, and 16
value = 0.25
MEMORY = 512, 1024, or 2048
value = 0.5
MEMORY = 1024, 2048, 3072, or 4096
value = 1
MEMORY = 2048, 3072, 4096, 5120, 6144, 7168, or 8192
value = 2
MEMORY = 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
value = 4
MEMORY = 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
value = 8
MEMORY = 16384, 20480, 24576, 28672, 32768, 36864, 40960, 45056, 49152, 53248, 57344, or 61440
value = 16
MEMORY = 32768, 40960, 49152, 57344, 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
type (string) --
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
linuxParameters (dict) --
Linux-specific modifications that are applied to the container, such as details for device mappings.
devices (list) --
Any of the host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
(dict) --
An object that represents a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
hostPath (string) --
The path for the device on the host container instance.
containerPath (string) --
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
permissions (list) --
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
(string) --
initProcessEnabled (boolean) --
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
sharedMemorySize (integer) --
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
tmpfs (list) --
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide this parameter for this resource type.
(dict) --
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
containerPath (string) --
The absolute file path in the container where the tmpfs volume is mounted.
size (integer) --
The size (in MiB) of the tmpfs volume.
mountOptions (list) --
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
(string) --
maxSwap (integer) --
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance that it's running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
swappiness (integer) --
You can use this parameter to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping to not occur unless absolutely necessary. A swappiness value of 100 causes pages to be swapped aggressively. Valid values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Swap space must be enabled and allocated on the container instance for the containers to use.
Note
By default, the Amazon ECS optimized AMIs don't have swap enabled. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
The swap space parameters are only supported for job definitions using EC2 resources.
If the maxSwap and swappiness parameters are omitted from a job definition, each container has a default swappiness value of 60. Moreover, the total swap usage is limited to two times the memory reservation of the container.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
logConfiguration (dict) --
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
logDriver (string) --
The log driver to use for the container. The valid values that are listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information including usage and options, see Fluentd logging driver in the Docker documentation .
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information including usage and options, see Graylog Extended Format logging driver in the Docker documentation .
journald
Specifies the journald logging driver. For more information including usage and options, see Journald logging driver in the Docker documentation .
json-file
Specifies the JSON file logging driver. For more information including usage and options, see JSON File logging driver in the Docker documentation .
splunk
Specifies the Splunk logging driver. For more information including usage and options, see Splunk logging driver in the Docker documentation .
syslog
Specifies the syslog logging driver. For more information including usage and options, see Syslog logging driver in the Docker documentation .
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
options (dict) --
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
(string) --
(string) --
secretOptions (list) --
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
secrets (list) --
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) --
The name of the secret.
valueFrom (string) --
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
networkConfiguration (dict) --
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
assignPublicIp (string) --
Indicates whether the job has a public IP address. For a job that's running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking in the Amazon Elastic Container Service Developer Guide . The default value is "DISABLED ".
fargatePlatformConfiguration (dict) --
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
platformVersion (string) --
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
ephemeralStorage (dict) --
The amount of ephemeral storage to allocate for the task. This parameter is used to expand the total amount of ephemeral storage available, beyond the default amount, for tasks hosted on Fargate.
sizeInGiB (integer) --
The total amount, in GiB, of ephemeral storage to set for the task. The minimum supported value is 21 GiB and the maximum supported value is 200 GiB.
arrayProperties (dict) --
The array properties of the job, if it's an array job.
statusSummary (dict) --
A summary of the number of array job children in each available job status. This parameter is returned for parent array jobs.
(string) --
(integer) --
size (integer) --
The size of the array job. This parameter is returned for parent array jobs.
index (integer) --
The job index within the array that's associated with this job. This parameter is returned for array job children.
timeout (dict) --
The timeout configuration for the job.
attemptDurationSeconds (integer) --
The job timeout time (in seconds) that's measured from the job attempt's startedAt timestamp. After this time passes, Batch terminates your jobs if they aren't finished. The minimum value for the timeout is 60 seconds.
For array jobs, the timeout applies to the child jobs, not to the parent array job.
For multi-node parallel (MNP) jobs, the timeout applies to the whole job, not to the individual nodes.
tags (dict) --
The tags that are applied to the job.
(string) --
(string) --
propagateTags (boolean) --
Specifies whether to propagate the tags from the job or job definition to the corresponding Amazon ECS task. If no value is specified, the tags aren't propagated. Tags can only be propagated to the tasks when the tasks are created. For tags with the same name, job tags are given priority over job definitions tags. If the total number of combined tags from the job and job definition is over 50, the job is moved to the FAILED state.
platformCapabilities (list) --
The platform capabilities required by the job definition. If no value is specified, it defaults to EC2 . Jobs run on Fargate resources specify FARGATE .
(string) --
eksProperties (dict) --
An object with various properties that are specific to Amazon EKS based jobs. Only one of container , eksProperties , or nodeDetails is specified.
podProperties (dict) --
The properties for the Kubernetes pod resources of a job.
serviceAccountName (string) --
The name of the service account that's used to run the pod. For more information, see Kubernetes service accounts and Configure a Kubernetes service account to assume an IAM role in the Amazon EKS User Guide and Configure service accounts for pods in the Kubernetes documentation .
hostNetwork (boolean) --
Indicates if the pod uses the hosts' network IP address. The default value is true . Setting this to false enables the Kubernetes pod networking model. Most Batch workloads are egress-only and don't require the overhead of IP allocation for each pod for incoming connections. For more information, see Host namespaces and Pod networking in the Kubernetes documentation .
dnsPolicy (string) --
The DNS policy for the pod. The default value is ClusterFirst . If the hostNetwork parameter is not specified, the default is ClusterFirstWithHostNet . ClusterFirst indicates that any DNS query that does not match the configured cluster domain suffix is forwarded to the upstream nameserver inherited from the node. If no value was specified for dnsPolicy in the RegisterJobDefinition API operation, then no value will be returned for dnsPolicy by either of DescribeJobDefinitions or DescribeJobs API operations. The pod spec setting will contain either ClusterFirst or ClusterFirstWithHostNet , depending on the value of the hostNetwork parameter. For more information, see Pod's DNS policy in the Kubernetes documentation .
Valid values: Default | ClusterFirst | ClusterFirstWithHostNet
containers (list) --
The properties of the container that's used on the Amazon EKS pod.
(dict) --
The details for container properties that are returned by DescribeJobs for jobs that use Amazon EKS.
name (string) --
The name of the container. If the name isn't specified, the default name "Default " is used. Each container in a pod must have a unique name.
image (string) --
The Docker image used to start the container.
imagePullPolicy (string) --
The image pull policy for the container. Supported values are Always , IfNotPresent , and Never . This parameter defaults to Always if the :latest tag is specified, IfNotPresent otherwise. For more information, see Updating images in the Kubernetes documentation .
command (list) --
The entrypoint for the container. For more information, see Entrypoint in the Kubernetes documentation .
(string) --
args (list) --
An array of arguments to the entrypoint. If this isn't specified, the CMD of the container image is used. This corresponds to the args member in the Entrypoint portion of the Pod in Kubernetes. Environment variable references are expanded using the container's environment.
If the referenced environment variable doesn't exist, the reference in the command isn't changed. For example, if the reference is to "$(NAME1) " and the NAME1 environment variable doesn't exist, the command string will remain "$(NAME1) ". $$ is replaced with $ and the resulting string isn't expanded. For example, $$(VAR_NAME) is passed as $(VAR_NAME) whether or not the VAR_NAME environment variable exists. For more information, see CMD in the Dockerfile reference and Define a command and arguments for a pod in the Kubernetes documentation .
(string) --
env (list) --
The environment variables to pass to a container.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
An environment variable.
name (string) --
The name of the environment variable.
value (string) --
The value of the environment variable.
resources (dict) --
The type and amount of resources to assign to a container. The supported resources include memory , cpu , and nvidia.com/gpu . For more information, see Resource management for pods and containers in the Kubernetes documentation .
limits (dict) --
The type and quantity of the resources to reserve for the container. The values vary based on the name that's specified. Resources can be requested using either the limits or the requests objects.
memory
The memory hard limit (in MiB) for the container, using whole integers, with a "Mi" suffix. If your container attempts to exceed the memory specified, the container is terminated. You must specify at least 4 MiB of memory for a job. memory can be specified in limits , requests , or both. If memory is specified in both places, then the value that's specified in limits must be equal to the value that's specified in requests .
Note
To maximize your resource utilization, provide your jobs with as much memory as possible for the specific instance type that you are using. To learn how, see Memory management in the Batch User Guide .
cpu
The number of CPUs that's reserved for the container. Values must be an even multiple of 0.25 . cpu can be specified in limits , requests , or both. If cpu is specified in both places, then the value that's specified in limits must be at least as large as the value that's specified in requests .
nvidia.com/gpu
The number of GPUs that's reserved for the container. Values must be a whole integer. memory can be specified in limits , requests , or both. If memory is specified in both places, then the value that's specified in limits must be equal to the value that's specified in requests .
(string) --
(string) --
requests (dict) --
The type and quantity of the resources to request for the container. The values vary based on the name that's specified. Resources can be requested by using either the limits or the requests objects.
memory
The memory hard limit (in MiB) for the container, using whole integers, with a "Mi" suffix. If your container attempts to exceed the memory specified, the container is terminated. You must specify at least 4 MiB of memory for a job. memory can be specified in limits , requests , or both. If memory is specified in both, then the value that's specified in limits must be equal to the value that's specified in requests .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
cpu
The number of CPUs that are reserved for the container. Values must be an even multiple of 0.25 . cpu can be specified in limits , requests , or both. If cpu is specified in both, then the value that's specified in limits must be at least as large as the value that's specified in requests .
nvidia.com/gpu
The number of GPUs that are reserved for the container. Values must be a whole integer. nvidia.com/gpu can be specified in limits , requests , or both. If nvidia.com/gpu is specified in both, then the value that's specified in limits must be equal to the value that's specified in requests .
(string) --
(string) --
exitCode (integer) --
The exit code for the job attempt. A non-zero exit code is considered failed.
reason (string) --
A short human-readable string to provide additional details for a running or stopped container. It can be up to 255 characters long.
volumeMounts (list) --
The volume mounts for the container. Batch supports emptyDir , hostPath , and secret volume types. For more information about volumes and volume mounts in Kubernetes, see Volumes in the Kubernetes documentation .
(dict) --
The volume mounts for a container for an Amazon EKS job. For more information about volumes and volume mounts in Kubernetes, see Volumes in the Kubernetes documentation .
name (string) --
The name the volume mount. This must match the name of one of the volumes in the pod.
mountPath (string) --
The path on the container where the volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
securityContext (dict) --
The security context for a job. For more information, see Configure a security context for a pod or container in the Kubernetes documentation .
runAsUser (integer) --
When this parameter is specified, the container is run as the specified user ID (uid ). If this parameter isn't specified, the default is the user that's specified in the image metadata. This parameter maps to RunAsUser and MustRanAs policy in the Users and groups pod security policies in the Kubernetes documentation .
runAsGroup (integer) --
When this parameter is specified, the container is run as the specified group ID (gid ). If this parameter isn't specified, the default is the group that's specified in the image metadata. This parameter maps to RunAsGroup and MustRunAs policy in the Users and groups pod security policies in the Kubernetes documentation .
privileged (boolean) --
When this parameter is true , the container is given elevated permissions on the host container instance. The level of permissions are similar to the root user permissions. The default value is false . This parameter maps to privileged policy in the Privileged pod security policies in the Kubernetes documentation .
readOnlyRootFilesystem (boolean) --
When this parameter is true , the container is given read-only access to its root file system. The default value is false . This parameter maps to ReadOnlyRootFilesystem policy in the Volumes and file systems pod security policies in the Kubernetes documentation .
runAsNonRoot (boolean) --
When this parameter is specified, the container is run as a user with a uid other than 0. If this parameter isn't specified, so such rule is enforced. This parameter maps to RunAsUser and MustRunAsNonRoot policy in the Users and groups pod security policies in the Kubernetes documentation .
volumes (list) --
Specifies the volumes for a job definition using Amazon EKS resources.
(dict) --
Specifies an Amazon EKS volume for a job definition.
name (string) --
The name of the volume. The name must be allowed as a DNS subdomain name. For more information, see DNS subdomain names in the Kubernetes documentation .
hostPath (dict) --
Specifies the configuration of a Kubernetes hostPath volume. For more information, see hostPath in the Kubernetes documentation .
path (string) --
The path of the file or directory on the host to mount into containers on the pod.
emptyDir (dict) --
Specifies the configuration of a Kubernetes emptyDir volume. For more information, see emptyDir in the Kubernetes documentation .
medium (string) --
The medium to store the volume. The default value is an empty string, which uses the storage of the node.
""
(Default) Use the disk storage of the node.
"Memory"
Use the tmpfs volume that's backed by the RAM of the node. Contents of the volume are lost when the node reboots, and any storage on the volume counts against the container's memory limit.
sizeLimit (string) --
The maximum size of the volume. By default, there's no maximum size defined.
secret (dict) --
Specifies the configuration of a Kubernetes secret volume. For more information, see secret in the Kubernetes documentation .
secretName (string) --
The name of the secret. The name must be allowed as a DNS subdomain name. For more information, see DNS subdomain names in the Kubernetes documentation .
optional (boolean) --
Specifies whether the secret or the secret's keys must be defined.
podName (string) --
The name of the pod for this job.
nodeName (string) --
The name of the node for this job.
eksAttempts (list) --
A list of job attempts that are associated with this job.
(dict) --
An object that represents the details of a job attempt for a job attempt by an Amazon EKS container.
containers (list) --
The details for the final status of the containers for this job attempt.
(dict) --
An object that represents the details for an attempt for a job attempt that an Amazon EKS container runs.
exitCode (integer) --
The exit code for the job attempt. A non-zero exit code is considered failed.
reason (string) --
A short (255 max characters) human-readable string to provide additional details for a running or stopped container.
podName (string) --
The name of the pod for this job attempt.
nodeName (string) --
The name of the node for this job attempt.
startedAt (integer) --
The Unix timestamp (in milliseconds) for when the attempt was started (when the attempt transitioned from the STARTING state to the RUNNING state).
stoppedAt (integer) --
The Unix timestamp (in milliseconds) for when the attempt was stopped. This happens when the attempt transitioned from the RUNNING state to a terminal state, such as SUCCEEDED or FAILED .
statusReason (string) --
A short, human-readable string to provide additional details for the current status of the job attempt.
isCancelled (boolean) --
Indicates whether the job is canceled.
isTerminated (boolean) --
Indicates whether the job is terminated.
{'containerProperties': {'ephemeralStorage': {'sizeInGiB': 'integer'}},
'eksProperties': {'podProperties': {'metadata': {'labels': {'string': 'string'}}}},
'nodeProperties': {'nodeRangeProperties': {'container': {'ephemeralStorage': {'sizeInGiB': 'integer'}}}}}
Registers an Batch job definition.
See also: AWS API Documentation
Request Syntax
client.register_job_definition(
jobDefinitionName='string',
type='container'|'multinode',
parameters={
'string': 'string'
},
schedulingPriority=123,
containerProperties={
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
},
'ephemeralStorage': {
'sizeInGiB': 123
}
},
nodeProperties={
'numNodes': 123,
'mainNode': 123,
'nodeRangeProperties': [
{
'targetNodes': 'string',
'container': {
'image': 'string',
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'jobRoleArn': 'string',
'executionRoleArn': 'string',
'volumes': [
{
'host': {
'sourcePath': 'string'
},
'name': 'string',
'efsVolumeConfiguration': {
'fileSystemId': 'string',
'rootDirectory': 'string',
'transitEncryption': 'ENABLED'|'DISABLED',
'transitEncryptionPort': 123,
'authorizationConfig': {
'accessPointId': 'string',
'iam': 'ENABLED'|'DISABLED'
}
}
},
],
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'mountPoints': [
{
'containerPath': 'string',
'readOnly': True|False,
'sourceVolume': 'string'
},
],
'readonlyRootFilesystem': True|False,
'privileged': True|False,
'ulimits': [
{
'hardLimit': 123,
'name': 'string',
'softLimit': 123
},
],
'user': 'string',
'instanceType': 'string',
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
],
'linuxParameters': {
'devices': [
{
'hostPath': 'string',
'containerPath': 'string',
'permissions': [
'READ'|'WRITE'|'MKNOD',
]
},
],
'initProcessEnabled': True|False,
'sharedMemorySize': 123,
'tmpfs': [
{
'containerPath': 'string',
'size': 123,
'mountOptions': [
'string',
]
},
],
'maxSwap': 123,
'swappiness': 123
},
'logConfiguration': {
'logDriver': 'json-file'|'syslog'|'journald'|'gelf'|'fluentd'|'awslogs'|'splunk',
'options': {
'string': 'string'
},
'secretOptions': [
{
'name': 'string',
'valueFrom': 'string'
},
]
},
'secrets': [
{
'name': 'string',
'valueFrom': 'string'
},
],
'networkConfiguration': {
'assignPublicIp': 'ENABLED'|'DISABLED'
},
'fargatePlatformConfiguration': {
'platformVersion': 'string'
},
'ephemeralStorage': {
'sizeInGiB': 123
}
}
},
]
},
retryStrategy={
'attempts': 123,
'evaluateOnExit': [
{
'onStatusReason': 'string',
'onReason': 'string',
'onExitCode': 'string',
'action': 'RETRY'|'EXIT'
},
]
},
propagateTags=True|False,
timeout={
'attemptDurationSeconds': 123
},
tags={
'string': 'string'
},
platformCapabilities=[
'EC2'|'FARGATE',
],
eksProperties={
'podProperties': {
'serviceAccountName': 'string',
'hostNetwork': True|False,
'dnsPolicy': 'string',
'containers': [
{
'name': 'string',
'image': 'string',
'imagePullPolicy': 'string',
'command': [
'string',
],
'args': [
'string',
],
'env': [
{
'name': 'string',
'value': 'string'
},
],
'resources': {
'limits': {
'string': 'string'
},
'requests': {
'string': 'string'
}
},
'volumeMounts': [
{
'name': 'string',
'mountPath': 'string',
'readOnly': True|False
},
],
'securityContext': {
'runAsUser': 123,
'runAsGroup': 123,
'privileged': True|False,
'readOnlyRootFilesystem': True|False,
'runAsNonRoot': True|False
}
},
],
'volumes': [
{
'name': 'string',
'hostPath': {
'path': 'string'
},
'emptyDir': {
'medium': 'string',
'sizeLimit': 'string'
},
'secret': {
'secretName': 'string',
'optional': True|False
}
},
],
'metadata': {
'labels': {
'string': 'string'
}
}
}
}
)
string
[REQUIRED]
The name of the job definition to register. It can be up to 128 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
string
[REQUIRED]
The type of job definition. For more information about multi-node parallel jobs, see Creating a multi-node parallel job definition in the Batch User Guide .
Note
If the job is run on Fargate resources, then multinode isn't supported.
dict
Default parameter substitution placeholders to set in the job definition. Parameters are specified as a key-value pair mapping. Parameters in a SubmitJob request override any corresponding parameter defaults from the job definition.
(string) --
(string) --
integer
The scheduling priority for jobs that are submitted with this job definition. This only affects jobs in job queues with a fair share policy. Jobs with a higher scheduling priority are scheduled before jobs with a lower scheduling priority.
The minimum supported value is 0 and the maximum supported value is 9999.
dict
An object with various properties specific to Amazon ECS based single-node container-based jobs. If the job definition's type parameter is container , then you must specify either containerProperties or nodeProperties . This must not be specified for Amazon EKS based job definitions.
Note
If the job runs on Fargate resources, then you must not specify nodeProperties ; use only containerProperties .
image (string) --
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . It can be 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), underscores (_), colons (:), periods (.), forward slashes (/), and number signs (#). This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
Images in Amazon ECR Public repositories use the full registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .
Images in Amazon ECR repositories use the full registry and repository URI (for example, 123456789012.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).
Images in official repositories on Docker Hub use a single name (for example, ubuntu or mongo ).
Images in other repositories on Docker Hub are qualified with an organization name (for example, amazon/amazon-ecs-agent ).
Images in other online repositories are qualified further by a domain name (for example, quay.io/assemblyline/ubuntu ).
vcpus (integer) --
This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
memory (integer) --
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs that run on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
command (list) --
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
(string) --
jobRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
executionRoleArn (string) --
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
volumes (list) --
A list of data volumes used in a job.
(dict) --
A data volume that's used in a job's container properties.
host (dict) --
The contents of the host parameter determine whether your data volume persists on the host container instance and where it's stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers that are associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
sourcePath (string) --
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources. Don't provide this for these jobs.
name (string) --
The name of the volume. It can be up to 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
efsVolumeConfiguration (dict) --
This parameter is specified when you're using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
fileSystemId (string) -- [REQUIRED]
The Amazon EFS file system ID to use.
rootDirectory (string) --
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
transitEncryption (string) --
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
transitEncryptionPort (integer) --
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
authorizationConfig (dict) --
The authorization configuration details for the Amazon EFS file system.
accessPointId (string) --
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which enforces the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
iam (string) --
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
environment (list) --
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
mountPoints (list) --
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
(dict) --
Details for a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
containerPath (string) --
The path on the container where the host volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
sourceVolume (string) --
The name of the volume to mount.
readonlyRootFilesystem (boolean) --
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
privileged (boolean) --
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
ulimits (list) --
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
hardLimit (integer) -- [REQUIRED]
The hard limit for the ulimit type.
name (string) -- [REQUIRED]
The type of the ulimit .
softLimit (integer) -- [REQUIRED]
The soft limit for the ulimit type.
user (string) --
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
instanceType (string) --
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
resourceRequirements (list) --
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) -- [REQUIRED]
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. Make sure that the number of GPUs reserved for all containers in a job doesn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs aren't available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU = 0.25
value = 1024
VCPU = 0.25 or 0.5
value = 2048
VCPU = 0.25, 0.5, or 1
value = 3072
VCPU = 0.5, or 1
value = 4096
VCPU = 0.5, 1, or 2
value = 5120, 6144, or 7168
VCPU = 1 or 2
value = 8192
VCPU = 1, 2, or 4
value = 9216, 10240, 11264, 12288, 13312, 14336, or 15360
VCPU = 2 or 4
value = 16384
VCPU = 2, 4, or 8
value = 17408, 18432, 19456, 21504, 22528, 23552, 25600, 26624, 27648, 29696, or 30720
VCPU = 4
value = 20480, 24576, or 28672
VCPU = 4 or 8
value = 36864, 45056, 53248, or 61440
VCPU = 8
value = 32768, 40960, 49152, or 57344
VCPU = 8 or 16
value = 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
VCPU = 16
type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
The default for the Fargate On-Demand vCPU resource count quota is 6 vCPUs. For more information about Fargate quotas, see Fargate quotas in the Amazon Web Services General Reference .
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, 4, 8, and 16
value = 0.25
MEMORY = 512, 1024, or 2048
value = 0.5
MEMORY = 1024, 2048, 3072, or 4096
value = 1
MEMORY = 2048, 3072, 4096, 5120, 6144, 7168, or 8192
value = 2
MEMORY = 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
value = 4
MEMORY = 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
value = 8
MEMORY = 16384, 20480, 24576, 28672, 32768, 36864, 40960, 45056, 49152, 53248, 57344, or 61440
value = 16
MEMORY = 32768, 40960, 49152, 57344, 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
type (string) -- [REQUIRED]
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
linuxParameters (dict) --
Linux-specific modifications that are applied to the container, such as details for device mappings.
devices (list) --
Any of the host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
(dict) --
An object that represents a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
hostPath (string) -- [REQUIRED]
The path for the device on the host container instance.
containerPath (string) --
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
permissions (list) --
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
(string) --
initProcessEnabled (boolean) --
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
sharedMemorySize (integer) --
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
tmpfs (list) --
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide this parameter for this resource type.
(dict) --
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
containerPath (string) -- [REQUIRED]
The absolute file path in the container where the tmpfs volume is mounted.
size (integer) -- [REQUIRED]
The size (in MiB) of the tmpfs volume.
mountOptions (list) --
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
(string) --
maxSwap (integer) --
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance that it's running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
swappiness (integer) --
You can use this parameter to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping to not occur unless absolutely necessary. A swappiness value of 100 causes pages to be swapped aggressively. Valid values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Swap space must be enabled and allocated on the container instance for the containers to use.
Note
By default, the Amazon ECS optimized AMIs don't have swap enabled. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
The swap space parameters are only supported for job definitions using EC2 resources.
If the maxSwap and swappiness parameters are omitted from a job definition, each container has a default swappiness value of 60. Moreover, the total swap usage is limited to two times the memory reservation of the container.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
logConfiguration (dict) --
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
logDriver (string) -- [REQUIRED]
The log driver to use for the container. The valid values that are listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information including usage and options, see Fluentd logging driver in the Docker documentation .
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information including usage and options, see Graylog Extended Format logging driver in the Docker documentation .
journald
Specifies the journald logging driver. For more information including usage and options, see Journald logging driver in the Docker documentation .
json-file
Specifies the JSON file logging driver. For more information including usage and options, see JSON File logging driver in the Docker documentation .
splunk
Specifies the Splunk logging driver. For more information including usage and options, see Splunk logging driver in the Docker documentation .
syslog
Specifies the syslog logging driver. For more information including usage and options, see Syslog logging driver in the Docker documentation .
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
options (dict) --
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
(string) --
(string) --
secretOptions (list) --
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) -- [REQUIRED]
The name of the secret.
valueFrom (string) -- [REQUIRED]
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
secrets (list) --
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) -- [REQUIRED]
The name of the secret.
valueFrom (string) -- [REQUIRED]
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
networkConfiguration (dict) --
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
assignPublicIp (string) --
Indicates whether the job has a public IP address. For a job that's running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking in the Amazon Elastic Container Service Developer Guide . The default value is "DISABLED ".
fargatePlatformConfiguration (dict) --
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
platformVersion (string) --
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
ephemeralStorage (dict) --
The amount of ephemeral storage to allocate for the task. This parameter is used to expand the total amount of ephemeral storage available, beyond the default amount, for tasks hosted on Fargate.
sizeInGiB (integer) -- [REQUIRED]
The total amount, in GiB, of ephemeral storage to set for the task. The minimum supported value is 21 GiB and the maximum supported value is 200 GiB.
dict
An object with various properties specific to multi-node parallel jobs. If you specify node properties for a job, it becomes a multi-node parallel job. For more information, see Multi-node Parallel Jobs in the Batch User Guide . If the job definition's type parameter is container , then you must specify either containerProperties or nodeProperties .
Note
If the job runs on Fargate resources, then you must not specify nodeProperties ; use containerProperties instead.
Note
If the job runs on Amazon EKS resources, then you must not specify nodeProperties .
numNodes (integer) -- [REQUIRED]
The number of nodes that are associated with a multi-node parallel job.
mainNode (integer) -- [REQUIRED]
Specifies the node index for the main node of a multi-node parallel job. This node index value must be fewer than the number of nodes.
nodeRangeProperties (list) -- [REQUIRED]
A list of node ranges and their properties that are associated with a multi-node parallel job.
(dict) --
An object that represents the properties of the node range for a multi-node parallel job.
targetNodes (string) -- [REQUIRED]
The range of nodes, using node index values. A range of 0:3 indicates nodes with index values of 0 through 3 . If the starting range value is omitted (:n ), then 0 is used to start the range. If the ending range value is omitted (n: ), then the highest possible node index is used to end the range. Your accumulative node ranges must account for all nodes (0:n ). You can nest node ranges (for example, 0:10 and 4:5 ). In this case, the 4:5 range properties override the 0:10 properties.
container (dict) --
The container details for the node range.
image (string) --
The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with `` repository-url /image :tag `` . It can be 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), underscores (_), colons (:), periods (.), forward slashes (/), and number signs (#). This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of docker run .
Note
Docker image architecture must match the processor architecture of the compute resources that they're scheduled on. For example, ARM-based Docker images can only run on ARM-based compute resources.
Images in Amazon ECR Public repositories use the full registry/repository[:tag] or registry/repository[@digest] naming conventions. For example, ``public.ecr.aws/registry_alias /my-web-app :latest `` .
Images in Amazon ECR repositories use the full registry and repository URI (for example, 123456789012.dkr.ecr.<region-name>.amazonaws.com/<repository-name> ).
Images in official repositories on Docker Hub use a single name (for example, ubuntu or mongo ).
Images in other repositories on Docker Hub are qualified with an organization name (for example, amazon/amazon-ecs-agent ).
Images in other online repositories are qualified further by a domain name (for example, quay.io/assemblyline/ubuntu ).
vcpus (integer) --
This parameter is deprecated, use resourceRequirements to specify the vCPU requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs running on EC2 resources, it specifies the number of vCPUs reserved for the job.
Each vCPU is equivalent to 1,024 CPU shares. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . The number of vCPUs must be specified but can be specified in several places. You must specify it at least once for each node.
memory (integer) --
This parameter is deprecated, use resourceRequirements to specify the memory requirements for the job definition. It's not supported for jobs running on Fargate resources. For jobs that run on EC2 resources, it specifies the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it's terminated. You must specify at least 4 MiB of memory for a job using this parameter. The memory hard limit can be specified in several places. It must be specified for each node at least once.
command (list) --
The command that's passed to the container. This parameter maps to Cmd in the Create a container section of the Docker Remote API and the COMMAND parameter to docker run . For more information, see https://docs.docker.com/engine/reference/builder/#cmd .
(string) --
jobRoleArn (string) --
The Amazon Resource Name (ARN) of the IAM role that the container can assume for Amazon Web Services permissions. For more information, see IAM roles for tasks in the Amazon Elastic Container Service Developer Guide .
executionRoleArn (string) --
The Amazon Resource Name (ARN) of the execution role that Batch can assume. For jobs that run on Fargate resources, you must provide an execution role. For more information, see Batch execution IAM role in the Batch User Guide .
volumes (list) --
A list of data volumes used in a job.
(dict) --
A data volume that's used in a job's container properties.
host (dict) --
The contents of the host parameter determine whether your data volume persists on the host container instance and where it's stored. If the host parameter is empty, then the Docker daemon assigns a host path for your data volume. However, the data isn't guaranteed to persist after the containers that are associated with it stop running.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
sourcePath (string) --
The path on the host container instance that's presented to the container. If this parameter is empty, then the Docker daemon has assigned a host path for you. If this parameter contains a file location, then the data volume persists at the specified location on the host container instance until you delete it manually. If the source path location doesn't exist on the host container instance, the Docker daemon creates it. If the location does exist, the contents of the source path folder are exported.
Note
This parameter isn't applicable to jobs that run on Fargate resources. Don't provide this for these jobs.
name (string) --
The name of the volume. It can be up to 255 characters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_). This name is referenced in the sourceVolume parameter of container definition mountPoints .
efsVolumeConfiguration (dict) --
This parameter is specified when you're using an Amazon Elastic File System file system for job storage. Jobs that are running on Fargate resources must specify a platformVersion of at least 1.4.0 .
fileSystemId (string) -- [REQUIRED]
The Amazon EFS file system ID to use.
rootDirectory (string) --
The directory within the Amazon EFS file system to mount as the root directory inside the host. If this parameter is omitted, the root of the Amazon EFS volume is used instead. Specifying / has the same effect as omitting this parameter. The maximum length is 4,096 characters.
Warning
If an EFS access point is specified in the authorizationConfig , the root directory parameter must either be omitted or set to / , which enforces the path set on the Amazon EFS access point.
transitEncryption (string) --
Determines whether to enable encryption for Amazon EFS data in transit between the Amazon ECS host and the Amazon EFS server. Transit encryption must be enabled if Amazon EFS IAM authorization is used. If this parameter is omitted, the default value of DISABLED is used. For more information, see Encrypting data in transit in the Amazon Elastic File System User Guide .
transitEncryptionPort (integer) --
The port to use when sending encrypted data between the Amazon ECS host and the Amazon EFS server. If you don't specify a transit encryption port, it uses the port selection strategy that the Amazon EFS mount helper uses. The value must be between 0 and 65,535. For more information, see EFS mount helper in the Amazon Elastic File System User Guide .
authorizationConfig (dict) --
The authorization configuration details for the Amazon EFS file system.
accessPointId (string) --
The Amazon EFS access point ID to use. If an access point is specified, the root directory value specified in the EFSVolumeConfiguration must either be omitted or set to / which enforces the path set on the EFS access point. If an access point is used, transit encryption must be enabled in the EFSVolumeConfiguration . For more information, see Working with Amazon EFS access points in the Amazon Elastic File System User Guide .
iam (string) --
Whether or not to use the Batch job IAM role defined in a job definition when mounting the Amazon EFS file system. If enabled, transit encryption must be enabled in the EFSVolumeConfiguration . If this parameter is omitted, the default value of DISABLED is used. For more information, see Using Amazon EFS access points in the Batch User Guide . EFS IAM authorization requires that TransitEncryption be ENABLED and that a JobRoleArn is specified.
environment (list) --
The environment variables to pass to a container. This parameter maps to Env in the Create a container section of the Docker Remote API and the --env option to docker run .
Warning
We don't recommend using plaintext environment variables for sensitive information, such as credential data.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
mountPoints (list) --
The mount points for data volumes in your container. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run .
(dict) --
Details for a Docker volume mount point that's used in a job's container properties. This parameter maps to Volumes in the Create a container section of the Docker Remote API and the --volume option to docker run.
containerPath (string) --
The path on the container where the host volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
sourceVolume (string) --
The name of the volume to mount.
readonlyRootFilesystem (boolean) --
When this parameter is true, the container is given read-only access to its root file system. This parameter maps to ReadonlyRootfs in the Create a container section of the Docker Remote API and the --read-only option to docker run .
privileged (boolean) --
When this parameter is true, the container is given elevated permissions on the host container instance (similar to the root user). This parameter maps to Privileged in the Create a container section of the Docker Remote API and the --privileged option to docker run . The default value is false.
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided, or specified as false.
ulimits (list) --
A list of ulimits to set in the container. This parameter maps to Ulimits in the Create a container section of the Docker Remote API and the --ulimit option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
(dict) --
The ulimit settings to pass to the container.
Note
This object isn't applicable to jobs that are running on Fargate resources.
hardLimit (integer) -- [REQUIRED]
The hard limit for the ulimit type.
name (string) -- [REQUIRED]
The type of the ulimit .
softLimit (integer) -- [REQUIRED]
The soft limit for the ulimit type.
user (string) --
The user name to use inside the container. This parameter maps to User in the Create a container section of the Docker Remote API and the --user option to docker run .
instanceType (string) --
The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
resourceRequirements (list) --
The type and amount of resources to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) -- [REQUIRED]
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. Make sure that the number of GPUs reserved for all containers in a job doesn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs aren't available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU = 0.25
value = 1024
VCPU = 0.25 or 0.5
value = 2048
VCPU = 0.25, 0.5, or 1
value = 3072
VCPU = 0.5, or 1
value = 4096
VCPU = 0.5, 1, or 2
value = 5120, 6144, or 7168
VCPU = 1 or 2
value = 8192
VCPU = 1, 2, or 4
value = 9216, 10240, 11264, 12288, 13312, 14336, or 15360
VCPU = 2 or 4
value = 16384
VCPU = 2, 4, or 8
value = 17408, 18432, 19456, 21504, 22528, 23552, 25600, 26624, 27648, 29696, or 30720
VCPU = 4
value = 20480, 24576, or 28672
VCPU = 4 or 8
value = 36864, 45056, 53248, or 61440
VCPU = 8
value = 32768, 40960, 49152, or 57344
VCPU = 8 or 16
value = 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
VCPU = 16
type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
The default for the Fargate On-Demand vCPU resource count quota is 6 vCPUs. For more information about Fargate quotas, see Fargate quotas in the Amazon Web Services General Reference .
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, 4, 8, and 16
value = 0.25
MEMORY = 512, 1024, or 2048
value = 0.5
MEMORY = 1024, 2048, 3072, or 4096
value = 1
MEMORY = 2048, 3072, 4096, 5120, 6144, 7168, or 8192
value = 2
MEMORY = 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
value = 4
MEMORY = 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
value = 8
MEMORY = 16384, 20480, 24576, 28672, 32768, 36864, 40960, 45056, 49152, 53248, 57344, or 61440
value = 16
MEMORY = 32768, 40960, 49152, 57344, 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
type (string) -- [REQUIRED]
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
linuxParameters (dict) --
Linux-specific modifications that are applied to the container, such as details for device mappings.
devices (list) --
Any of the host devices to expose to the container. This parameter maps to Devices in the Create a container section of the Docker Remote API and the --device option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
(dict) --
An object that represents a container instance host device.
Note
This object isn't applicable to jobs that are running on Fargate resources and shouldn't be provided.
hostPath (string) -- [REQUIRED]
The path for the device on the host container instance.
containerPath (string) --
The path inside the container that's used to expose the host device. By default, the hostPath value is used.
permissions (list) --
The explicit permissions to provide to the container for the device. By default, the container has permissions for read , write , and mknod for the device.
(string) --
initProcessEnabled (boolean) --
If true, run an init process inside the container that forwards signals and reaps processes. This parameter maps to the --init option to docker run . This parameter requires version 1.25 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
sharedMemorySize (integer) --
The value for the size (in MiB) of the /dev/shm volume. This parameter maps to the --shm-size option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
tmpfs (list) --
The container path, mount options, and size (in MiB) of the tmpfs mount. This parameter maps to the --tmpfs option to docker run .
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide this parameter for this resource type.
(dict) --
The container path, mount options, and size of the tmpfs mount.
Note
This object isn't applicable to jobs that are running on Fargate resources.
containerPath (string) -- [REQUIRED]
The absolute file path in the container where the tmpfs volume is mounted.
size (integer) -- [REQUIRED]
The size (in MiB) of the tmpfs volume.
mountOptions (list) --
The list of tmpfs volume mount options.
Valid values: "defaults " | "ro " | "rw " | "suid " | "nosuid " | "dev " | "nodev " | "exec " | "noexec " | "sync " | "async " | "dirsync " | "remount " | "mand " | "nomand " | "atime " | "noatime " | "diratime " | "nodiratime " | "bind " | "rbind" | "unbindable" | "runbindable" | "private" | "rprivate" | "shared" | "rshared" | "slave" | "rslave" | "relatime " | "norelatime " | "strictatime " | "nostrictatime " | "mode " | "uid " | "gid " | "nr_inodes " | "nr_blocks " | "mpol "
(string) --
maxSwap (integer) --
The total amount of swap memory (in MiB) a container can use. This parameter is translated to the --memory-swap option to docker run where the value is the sum of the container memory plus the maxSwap value. For more information, see ` --memory-swap details <https://docs.docker.com/config/containers/resource_constraints/#--memory-swap-details>`__ in the Docker documentation.
If a maxSwap value of 0 is specified, the container doesn't use swap. Accepted values are 0 or any positive integer. If the maxSwap parameter is omitted, the container doesn't use the swap configuration for the container instance that it's running on. A maxSwap value must be set for the swappiness parameter to be used.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
swappiness (integer) --
You can use this parameter to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping to not occur unless absolutely necessary. A swappiness value of 100 causes pages to be swapped aggressively. Valid values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used. If a value isn't specified for maxSwap , then this parameter is ignored. If maxSwap is set to 0, the container doesn't use swap. This parameter maps to the --memory-swappiness option to docker run .
Consider the following when you use a per-container swap configuration.
Swap space must be enabled and allocated on the container instance for the containers to use.
Note
By default, the Amazon ECS optimized AMIs don't have swap enabled. You must enable swap on the instance to use this feature. For more information, see Instance store swap volumes in the Amazon EC2 User Guide for Linux Instances or How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?
The swap space parameters are only supported for job definitions using EC2 resources.
If the maxSwap and swappiness parameters are omitted from a job definition, each container has a default swappiness value of 60. Moreover, the total swap usage is limited to two times the memory reservation of the container.
Note
This parameter isn't applicable to jobs that are running on Fargate resources. Don't provide it for these jobs.
logConfiguration (dict) --
The log configuration specification for the container.
This parameter maps to LogConfig in the Create a container section of the Docker Remote API and the --log-driver option to docker run . By default, containers use the same logging driver that the Docker daemon uses. However the container might use a different logging driver than the Docker daemon by specifying a log driver with this parameter in the container definition. To use a different logging driver for a container, the log system must be configured properly on the container instance (or on a different log server for remote logging options). For more information on the options for different supported log drivers, see Configure logging drivers in the Docker documentation.
Note
Batch currently supports a subset of the logging drivers available to the Docker daemon (shown in the LogConfiguration data type).
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
Note
The Amazon ECS container agent running on a container instance must register the logging drivers available on that instance with the ECS_AVAILABLE_LOGGING_DRIVERS environment variable before containers placed on that instance can use these log configuration options. For more information, see Amazon ECS container agent configuration in the Amazon Elastic Container Service Developer Guide .
logDriver (string) -- [REQUIRED]
The log driver to use for the container. The valid values that are listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are awslogs , fluentd , gelf , json-file , journald , logentries , syslog , and splunk .
Note
Jobs that are running on Fargate resources are restricted to the awslogs and splunk log drivers.
awslogs
Specifies the Amazon CloudWatch Logs logging driver. For more information, see Using the awslogs log driver in the Batch User Guide and Amazon CloudWatch Logs logging driver in the Docker documentation.
fluentd
Specifies the Fluentd logging driver. For more information including usage and options, see Fluentd logging driver in the Docker documentation .
gelf
Specifies the Graylog Extended Format (GELF) logging driver. For more information including usage and options, see Graylog Extended Format logging driver in the Docker documentation .
journald
Specifies the journald logging driver. For more information including usage and options, see Journald logging driver in the Docker documentation .
json-file
Specifies the JSON file logging driver. For more information including usage and options, see JSON File logging driver in the Docker documentation .
splunk
Specifies the Splunk logging driver. For more information including usage and options, see Splunk logging driver in the Docker documentation .
syslog
Specifies the syslog logging driver. For more information including usage and options, see Syslog logging driver in the Docker documentation .
Note
If you have a custom driver that's not listed earlier that you want to work with the Amazon ECS container agent, you can fork the Amazon ECS container agent project that's available on GitHub and customize it to work with that driver. We encourage you to submit pull requests for changes that you want to have included. However, Amazon Web Services doesn't currently support running modified copies of this software.
This parameter requires version 1.18 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
options (dict) --
The configuration options to send to the log driver. This parameter requires version 1.19 of the Docker Remote API or greater on your container instance. To check the Docker Remote API version on your container instance, log in to your container instance and run the following command: sudo docker version | grep "Server API version"
(string) --
(string) --
secretOptions (list) --
The secrets to pass to the log configuration. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) -- [REQUIRED]
The name of the secret.
valueFrom (string) -- [REQUIRED]
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
secrets (list) --
The secrets for the container. For more information, see Specifying sensitive data in the Batch User Guide .
(dict) --
An object that represents the secret to expose to your container. Secrets can be exposed to a container in the following ways:
To inject sensitive data into your containers as environment variables, use the secrets container definition parameter.
To reference sensitive information in the log configuration of a container, use the secretOptions container definition parameter.
For more information, see Specifying sensitive data in the Batch User Guide .
name (string) -- [REQUIRED]
The name of the secret.
valueFrom (string) -- [REQUIRED]
The secret to expose to the container. The supported values are either the full Amazon Resource Name (ARN) of the Secrets Manager secret or the full ARN of the parameter in the Amazon Web Services Systems Manager Parameter Store.
Note
If the Amazon Web Services Systems Manager Parameter Store parameter exists in the same Region as the job you're launching, then you can use either the full Amazon Resource Name (ARN) or name of the parameter. If the parameter exists in a different Region, then the full ARN must be specified.
networkConfiguration (dict) --
The network configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
assignPublicIp (string) --
Indicates whether the job has a public IP address. For a job that's running on Fargate resources in a private subnet to send outbound traffic to the internet (for example, to pull container images), the private subnet requires a NAT gateway be attached to route requests to the internet. For more information, see Amazon ECS task networking in the Amazon Elastic Container Service Developer Guide . The default value is "DISABLED ".
fargatePlatformConfiguration (dict) --
The platform configuration for jobs that are running on Fargate resources. Jobs that are running on EC2 resources must not specify this parameter.
platformVersion (string) --
The Fargate platform version where the jobs are running. A platform version is specified only for jobs that are running on Fargate resources. If one isn't specified, the LATEST platform version is used by default. This uses a recent, approved version of the Fargate platform for compute resources. For more information, see Fargate platform versions in the Amazon Elastic Container Service Developer Guide .
ephemeralStorage (dict) --
The amount of ephemeral storage to allocate for the task. This parameter is used to expand the total amount of ephemeral storage available, beyond the default amount, for tasks hosted on Fargate.
sizeInGiB (integer) -- [REQUIRED]
The total amount, in GiB, of ephemeral storage to set for the task. The minimum supported value is 21 GiB and the maximum supported value is 200 GiB.
dict
The retry strategy to use for failed jobs that are submitted with this job definition. Any retry strategy that's specified during a SubmitJob operation overrides the retry strategy defined here. If a job is terminated due to a timeout, it isn't retried.
attempts (integer) --
The number of times to move a job to the RUNNABLE status. You can specify between 1 and 10 attempts. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.
evaluateOnExit (list) --
Array of up to 5 objects that specify the conditions where jobs are retried or failed. If this parameter is specified, then the attempts parameter must also be specified. If none of the listed conditions match, then the job is retried.
(dict) --
Specifies an array of up to 5 conditions to be met, and an action to take (RETRY or EXIT ) if all conditions are met. If none of the EvaluateOnExit conditions in a RetryStrategy match, then the job is retried.
onStatusReason (string) --
Contains a glob pattern to match against the StatusReason returned for a job. The pattern can contain up to 512 characters. It can contain letters, numbers, periods (.), colons (:), and white spaces (including spaces or tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
onReason (string) --
Contains a glob pattern to match against the Reason returned for a job. The pattern can contain up to 512 characters. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces and tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
onExitCode (string) --
Contains a glob pattern to match against the decimal representation of the ExitCode returned for a job. The pattern can be up to 512 characters long. It can contain only numbers, and can end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can contain up to 512 characters.
action (string) -- [REQUIRED]
Specifies the action to take if all of the specified conditions (onStatusReason , onReason , and onExitCode ) are met. The values aren't case sensitive.
boolean
Specifies whether to propagate the tags from the job or job definition to the corresponding Amazon ECS task. If no value is specified, the tags are not propagated. Tags can only be propagated to the tasks during task creation. For tags with the same name, job tags are given priority over job definitions tags. If the total number of combined tags from the job and job definition is over 50, the job is moved to the FAILED state.
Note
If the job runs on Amazon EKS resources, then you must not specify propagateTags .
dict
The timeout configuration for jobs that are submitted with this job definition, after which Batch terminates your jobs if they have not finished. If a job is terminated due to a timeout, it isn't retried. The minimum value for the timeout is 60 seconds. Any timeout configuration that's specified during a SubmitJob operation overrides the timeout configuration defined here. For more information, see Job Timeouts in the Batch User Guide .
attemptDurationSeconds (integer) --
The job timeout time (in seconds) that's measured from the job attempt's startedAt timestamp. After this time passes, Batch terminates your jobs if they aren't finished. The minimum value for the timeout is 60 seconds.
For array jobs, the timeout applies to the child jobs, not to the parent array job.
For multi-node parallel (MNP) jobs, the timeout applies to the whole job, not to the individual nodes.
dict
The tags that you apply to the job definition to help you categorize and organize your resources. Each tag consists of a key and an optional value. For more information, see Tagging Amazon Web Services Resources in Batch User Guide .
(string) --
(string) --
list
The platform capabilities required by the job definition. If no value is specified, it defaults to EC2 . To run the job on Fargate resources, specify FARGATE .
Note
If the job runs on Amazon EKS resources, then you must not specify platformCapabilities .
(string) --
dict
An object with various properties that are specific to Amazon EKS based jobs. This must not be specified for Amazon ECS based job definitions.
podProperties (dict) --
The properties for the Kubernetes pod resources of a job.
serviceAccountName (string) --
The name of the service account that's used to run the pod. For more information, see Kubernetes service accounts and Configure a Kubernetes service account to assume an IAM role in the Amazon EKS User Guide and Configure service accounts for pods in the Kubernetes documentation .
hostNetwork (boolean) --
Indicates if the pod uses the hosts' network IP address. The default value is true . Setting this to false enables the Kubernetes pod networking model. Most Batch workloads are egress-only and don't require the overhead of IP allocation for each pod for incoming connections. For more information, see Host namespaces and Pod networking in the Kubernetes documentation .
dnsPolicy (string) --
The DNS policy for the pod. The default value is ClusterFirst . If the hostNetwork parameter is not specified, the default is ClusterFirstWithHostNet . ClusterFirst indicates that any DNS query that does not match the configured cluster domain suffix is forwarded to the upstream nameserver inherited from the node. For more information, see Pod's DNS policy in the Kubernetes documentation .
Valid values: Default | ClusterFirst | ClusterFirstWithHostNet
containers (list) --
The properties of the container that's used on the Amazon EKS pod.
(dict) --
EKS container properties are used in job definitions for Amazon EKS based job definitions to describe the properties for a container node in the pod that's launched as part of a job. This can't be specified for Amazon ECS based job definitions.
name (string) --
The name of the container. If the name isn't specified, the default name "Default " is used. Each container in a pod must have a unique name.
image (string) -- [REQUIRED]
The Docker image used to start the container.
imagePullPolicy (string) --
The image pull policy for the container. Supported values are Always , IfNotPresent , and Never . This parameter defaults to IfNotPresent . However, if the :latest tag is specified, it defaults to Always . For more information, see Updating images in the Kubernetes documentation .
command (list) --
The entrypoint for the container. This isn't run within a shell. If this isn't specified, the ENTRYPOINT of the container image is used. Environment variable references are expanded using the container's environment.
If the referenced environment variable doesn't exist, the reference in the command isn't changed. For example, if the reference is to "$(NAME1) " and the NAME1 environment variable doesn't exist, the command string will remain "$(NAME1) ." $$ is replaced with $ and the resulting string isn't expanded. For example, $$(VAR_NAME) will be passed as $(VAR_NAME) whether or not the VAR_NAME environment variable exists. The entrypoint can't be updated. For more information, see ENTRYPOINT in the Dockerfile reference and Define a command and arguments for a container and Entrypoint in the Kubernetes documentation .
(string) --
args (list) --
An array of arguments to the entrypoint. If this isn't specified, the CMD of the container image is used. This corresponds to the args member in the Entrypoint portion of the Pod in Kubernetes. Environment variable references are expanded using the container's environment.
If the referenced environment variable doesn't exist, the reference in the command isn't changed. For example, if the reference is to "$(NAME1) " and the NAME1 environment variable doesn't exist, the command string will remain "$(NAME1) ." $$ is replaced with $ , and the resulting string isn't expanded. For example, $$(VAR_NAME) is passed as $(VAR_NAME) whether or not the VAR_NAME environment variable exists. For more information, see CMD in the Dockerfile reference and Define a command and arguments for a pod in the Kubernetes documentation .
(string) --
env (list) --
The environment variables to pass to a container.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
An environment variable.
name (string) -- [REQUIRED]
The name of the environment variable.
value (string) --
The value of the environment variable.
resources (dict) --
The type and amount of resources to assign to a container. The supported resources include memory , cpu , and nvidia.com/gpu . For more information, see Resource management for pods and containers in the Kubernetes documentation .
limits (dict) --
The type and quantity of the resources to reserve for the container. The values vary based on the name that's specified. Resources can be requested using either the limits or the requests objects.
memory
The memory hard limit (in MiB) for the container, using whole integers, with a "Mi" suffix. If your container attempts to exceed the memory specified, the container is terminated. You must specify at least 4 MiB of memory for a job. memory can be specified in limits , requests , or both. If memory is specified in both places, then the value that's specified in limits must be equal to the value that's specified in requests .
Note
To maximize your resource utilization, provide your jobs with as much memory as possible for the specific instance type that you are using. To learn how, see Memory management in the Batch User Guide .
cpu
The number of CPUs that's reserved for the container. Values must be an even multiple of 0.25 . cpu can be specified in limits , requests , or both. If cpu is specified in both places, then the value that's specified in limits must be at least as large as the value that's specified in requests .
nvidia.com/gpu
The number of GPUs that's reserved for the container. Values must be a whole integer. memory can be specified in limits , requests , or both. If memory is specified in both places, then the value that's specified in limits must be equal to the value that's specified in requests .
(string) --
(string) --
requests (dict) --
The type and quantity of the resources to request for the container. The values vary based on the name that's specified. Resources can be requested by using either the limits or the requests objects.
memory
The memory hard limit (in MiB) for the container, using whole integers, with a "Mi" suffix. If your container attempts to exceed the memory specified, the container is terminated. You must specify at least 4 MiB of memory for a job. memory can be specified in limits , requests , or both. If memory is specified in both, then the value that's specified in limits must be equal to the value that's specified in requests .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
cpu
The number of CPUs that are reserved for the container. Values must be an even multiple of 0.25 . cpu can be specified in limits , requests , or both. If cpu is specified in both, then the value that's specified in limits must be at least as large as the value that's specified in requests .
nvidia.com/gpu
The number of GPUs that are reserved for the container. Values must be a whole integer. nvidia.com/gpu can be specified in limits , requests , or both. If nvidia.com/gpu is specified in both, then the value that's specified in limits must be equal to the value that's specified in requests .
(string) --
(string) --
volumeMounts (list) --
The volume mounts for the container. Batch supports emptyDir , hostPath , and secret volume types. For more information about volumes and volume mounts in Kubernetes, see Volumes in the Kubernetes documentation .
(dict) --
The volume mounts for a container for an Amazon EKS job. For more information about volumes and volume mounts in Kubernetes, see Volumes in the Kubernetes documentation .
name (string) --
The name the volume mount. This must match the name of one of the volumes in the pod.
mountPath (string) --
The path on the container where the volume is mounted.
readOnly (boolean) --
If this value is true , the container has read-only access to the volume. Otherwise, the container can write to the volume. The default value is false .
securityContext (dict) --
The security context for a job. For more information, see Configure a security context for a pod or container in the Kubernetes documentation .
runAsUser (integer) --
When this parameter is specified, the container is run as the specified user ID (uid ). If this parameter isn't specified, the default is the user that's specified in the image metadata. This parameter maps to RunAsUser and MustRanAs policy in the Users and groups pod security policies in the Kubernetes documentation .
runAsGroup (integer) --
When this parameter is specified, the container is run as the specified group ID (gid ). If this parameter isn't specified, the default is the group that's specified in the image metadata. This parameter maps to RunAsGroup and MustRunAs policy in the Users and groups pod security policies in the Kubernetes documentation .
privileged (boolean) --
When this parameter is true , the container is given elevated permissions on the host container instance. The level of permissions are similar to the root user permissions. The default value is false . This parameter maps to privileged policy in the Privileged pod security policies in the Kubernetes documentation .
readOnlyRootFilesystem (boolean) --
When this parameter is true , the container is given read-only access to its root file system. The default value is false . This parameter maps to ReadOnlyRootFilesystem policy in the Volumes and file systems pod security policies in the Kubernetes documentation .
runAsNonRoot (boolean) --
When this parameter is specified, the container is run as a user with a uid other than 0. If this parameter isn't specified, so such rule is enforced. This parameter maps to RunAsUser and MustRunAsNonRoot policy in the Users and groups pod security policies in the Kubernetes documentation .
volumes (list) --
Specifies the volumes for a job definition that uses Amazon EKS resources.
(dict) --
Specifies an Amazon EKS volume for a job definition.
name (string) -- [REQUIRED]
The name of the volume. The name must be allowed as a DNS subdomain name. For more information, see DNS subdomain names in the Kubernetes documentation .
hostPath (dict) --
Specifies the configuration of a Kubernetes hostPath volume. For more information, see hostPath in the Kubernetes documentation .
path (string) --
The path of the file or directory on the host to mount into containers on the pod.
emptyDir (dict) --
Specifies the configuration of a Kubernetes emptyDir volume. For more information, see emptyDir in the Kubernetes documentation .
medium (string) --
The medium to store the volume. The default value is an empty string, which uses the storage of the node.
""
(Default) Use the disk storage of the node.
"Memory"
Use the tmpfs volume that's backed by the RAM of the node. Contents of the volume are lost when the node reboots, and any storage on the volume counts against the container's memory limit.
sizeLimit (string) --
The maximum size of the volume. By default, there's no maximum size defined.
secret (dict) --
Specifies the configuration of a Kubernetes secret volume. For more information, see secret in the Kubernetes documentation .
secretName (string) -- [REQUIRED]
The name of the secret. The name must be allowed as a DNS subdomain name. For more information, see DNS subdomain names in the Kubernetes documentation .
optional (boolean) --
Specifies whether the secret or the secret's keys must be defined.
metadata (dict) --
labels (dict) --
(string) --
(string) --
dict
Response Syntax
{
'jobDefinitionName': 'string',
'jobDefinitionArn': 'string',
'revision': 123
}
Response Structure
(dict) --
jobDefinitionName (string) --
The name of the job definition.
jobDefinitionArn (string) --
The Amazon Resource Name (ARN) of the job definition.
revision (integer) --
The revision of the job definition.
{'eksPropertiesOverride': {'podProperties': {'metadata': {'labels': {'string': 'string'}}}}}
Submits an Batch job from a job definition. Parameters that are specified during SubmitJob override parameters defined in the job definition. vCPU and memory requirements that are specified in the resourceRequirements objects in the job definition are the exception. They can't be overridden this way using the memory and vcpus parameters. Rather, you must specify updates to job definition parameters in a resourceRequirements object that's included in the containerOverrides parameter.
Note
Job queues with a scheduling policy are limited to 500 active fair share identifiers at a time.
Warning
Jobs that run on Fargate resources can't be guaranteed to run for more than 14 days. This is because, after 14 days, Fargate resources might become unavailable and job might be terminated.
See also: AWS API Documentation
Request Syntax
client.submit_job(
jobName='string',
jobQueue='string',
shareIdentifier='string',
schedulingPriorityOverride=123,
arrayProperties={
'size': 123
},
dependsOn=[
{
'jobId': 'string',
'type': 'N_TO_N'|'SEQUENTIAL'
},
],
jobDefinition='string',
parameters={
'string': 'string'
},
containerOverrides={
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'instanceType': 'string',
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
]
},
nodeOverrides={
'numNodes': 123,
'nodePropertyOverrides': [
{
'targetNodes': 'string',
'containerOverrides': {
'vcpus': 123,
'memory': 123,
'command': [
'string',
],
'instanceType': 'string',
'environment': [
{
'name': 'string',
'value': 'string'
},
],
'resourceRequirements': [
{
'value': 'string',
'type': 'GPU'|'VCPU'|'MEMORY'
},
]
}
},
]
},
retryStrategy={
'attempts': 123,
'evaluateOnExit': [
{
'onStatusReason': 'string',
'onReason': 'string',
'onExitCode': 'string',
'action': 'RETRY'|'EXIT'
},
]
},
propagateTags=True|False,
timeout={
'attemptDurationSeconds': 123
},
tags={
'string': 'string'
},
eksPropertiesOverride={
'podProperties': {
'containers': [
{
'image': 'string',
'command': [
'string',
],
'args': [
'string',
],
'env': [
{
'name': 'string',
'value': 'string'
},
],
'resources': {
'limits': {
'string': 'string'
},
'requests': {
'string': 'string'
}
}
},
],
'metadata': {
'labels': {
'string': 'string'
}
}
}
}
)
string
[REQUIRED]
The name of the job. It can be up to 128 letters long. The first character must be alphanumeric, can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_).
string
[REQUIRED]
The job queue where the job is submitted. You can specify either the name or the Amazon Resource Name (ARN) of the queue.
string
The share identifier for the job. If the job queue doesn't have a scheduling policy, then this parameter must not be specified. If the job queue has a scheduling policy, then this parameter must be specified.
integer
The scheduling priority for the job. This only affects jobs in job queues with a fair share policy. Jobs with a higher scheduling priority are scheduled before jobs with a lower scheduling priority. This overrides any scheduling priority in the job definition.
The minimum supported value is 0 and the maximum supported value is 9999.
dict
The array properties for the submitted job, such as the size of the array. The array size can be between 2 and 10,000. If you specify array properties for a job, it becomes an array job. For more information, see Array Jobs in the Batch User Guide .
size (integer) --
The size of the array job.
list
A list of dependencies for the job. A job can depend upon a maximum of 20 jobs. You can specify a SEQUENTIAL type dependency without specifying a job ID for array jobs so that each child array job completes sequentially, starting at index 0. You can also specify an N_TO_N type dependency with a job ID for array jobs. In that case, each index child of this job must wait for the corresponding index child of each dependency to complete before it can begin.
(dict) --
An object that represents an Batch job dependency.
jobId (string) --
The job ID of the Batch job that's associated with this dependency.
type (string) --
The type of the job dependency.
string
[REQUIRED]
The job definition used by this job. This value can be one of definition-name , definition-name:revision , or the Amazon Resource Name (ARN) for the job definition, with or without the revision (``arn:aws:batch:region :account :job-definition/definition-name :revision `` , or ``arn:aws:batch:region :account :job-definition/definition-name `` ).
If the revision is not specified, then the latest active revision is used.
dict
Additional parameters passed to the job that replace parameter substitution placeholders that are set in the job definition. Parameters are specified as a key and value pair mapping. Parameters in a SubmitJob request override any corresponding parameter defaults from the job definition.
(string) --
(string) --
dict
An object with various properties that override the defaults for the job definition that specify the name of a container in the specified job definition and the overrides it should receive. You can override the default command for a container, which is specified in the job definition or the Docker image, with a command override. You can also override existing environment variables on a container or add new environment variables to it with an environment override.
vcpus (integer) --
This parameter is deprecated, use resourceRequirements to override the vcpus parameter that's set in the job definition. It's not supported for jobs running on Fargate resources. For jobs that run on EC2 resources, it overrides the vcpus parameter set in the job definition, but doesn't override any vCPU requirement specified in the resourceRequirements structure in the job definition. To override vCPU requirements that are specified in the resourceRequirements structure in the job definition, resourceRequirements must be specified in the SubmitJob request, with type set to VCPU and value set to the new value. For more information, see Can't override job definition resource requirements in the Batch User Guide .
memory (integer) --
This parameter is deprecated, use resourceRequirements to override the memory requirements specified in the job definition. It's not supported for jobs running on Fargate resources. For jobs that run on EC2 resources, it overrides the memory parameter set in the job definition, but doesn't override any memory requirement that's specified in the resourceRequirements structure in the job definition. To override memory requirements that are specified in the resourceRequirements structure in the job definition, resourceRequirements must be specified in the SubmitJob request, with type set to MEMORY and value set to the new value. For more information, see Can't override job definition resource requirements in the Batch User Guide .
command (list) --
The command to send to the container that overrides the default command from the Docker image or the job definition.
(string) --
instanceType (string) --
The instance type to use for a multi-node parallel job.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
environment (list) --
The environment variables to send to the container. You can add new environment variables, which are added to the container at launch, or you can override the existing environment variables from the Docker image or the job definition.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
resourceRequirements (list) --
The type and amount of resources to assign to a container. This overrides the settings in the job definition. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) -- [REQUIRED]
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. Make sure that the number of GPUs reserved for all containers in a job doesn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs aren't available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU = 0.25
value = 1024
VCPU = 0.25 or 0.5
value = 2048
VCPU = 0.25, 0.5, or 1
value = 3072
VCPU = 0.5, or 1
value = 4096
VCPU = 0.5, 1, or 2
value = 5120, 6144, or 7168
VCPU = 1 or 2
value = 8192
VCPU = 1, 2, or 4
value = 9216, 10240, 11264, 12288, 13312, 14336, or 15360
VCPU = 2 or 4
value = 16384
VCPU = 2, 4, or 8
value = 17408, 18432, 19456, 21504, 22528, 23552, 25600, 26624, 27648, 29696, or 30720
VCPU = 4
value = 20480, 24576, or 28672
VCPU = 4 or 8
value = 36864, 45056, 53248, or 61440
VCPU = 8
value = 32768, 40960, 49152, or 57344
VCPU = 8 or 16
value = 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
VCPU = 16
type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
The default for the Fargate On-Demand vCPU resource count quota is 6 vCPUs. For more information about Fargate quotas, see Fargate quotas in the Amazon Web Services General Reference .
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, 4, 8, and 16
value = 0.25
MEMORY = 512, 1024, or 2048
value = 0.5
MEMORY = 1024, 2048, 3072, or 4096
value = 1
MEMORY = 2048, 3072, 4096, 5120, 6144, 7168, or 8192
value = 2
MEMORY = 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
value = 4
MEMORY = 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
value = 8
MEMORY = 16384, 20480, 24576, 28672, 32768, 36864, 40960, 45056, 49152, 53248, 57344, or 61440
value = 16
MEMORY = 32768, 40960, 49152, 57344, 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
type (string) -- [REQUIRED]
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
dict
A list of node overrides in JSON format that specify the node range to target and the container overrides for that node range.
Note
This parameter isn't applicable to jobs that are running on Fargate resources; use containerOverrides instead.
numNodes (integer) --
The number of nodes to use with a multi-node parallel job. This value overrides the number of nodes that are specified in the job definition. To use this override, you must meet the following conditions:
There must be at least one node range in your job definition that has an open upper boundary, such as : or n: .
The lower boundary of the node range that's specified in the job definition must be fewer than the number of nodes specified in the override.
The main node index that's specified in the job definition must be fewer than the number of nodes specified in the override.
nodePropertyOverrides (list) --
The node property overrides for the job.
(dict) --
The object that represents any node overrides to a job definition that's used in a SubmitJob API operation.
targetNodes (string) -- [REQUIRED]
The range of nodes, using node index values, that's used to override. A range of 0:3 indicates nodes with index values of 0 through 3 . If the starting range value is omitted (:n ), then 0 is used to start the range. If the ending range value is omitted (n: ), then the highest possible node index is used to end the range.
containerOverrides (dict) --
The overrides that are sent to a node range.
vcpus (integer) --
This parameter is deprecated, use resourceRequirements to override the vcpus parameter that's set in the job definition. It's not supported for jobs running on Fargate resources. For jobs that run on EC2 resources, it overrides the vcpus parameter set in the job definition, but doesn't override any vCPU requirement specified in the resourceRequirements structure in the job definition. To override vCPU requirements that are specified in the resourceRequirements structure in the job definition, resourceRequirements must be specified in the SubmitJob request, with type set to VCPU and value set to the new value. For more information, see Can't override job definition resource requirements in the Batch User Guide .
memory (integer) --
This parameter is deprecated, use resourceRequirements to override the memory requirements specified in the job definition. It's not supported for jobs running on Fargate resources. For jobs that run on EC2 resources, it overrides the memory parameter set in the job definition, but doesn't override any memory requirement that's specified in the resourceRequirements structure in the job definition. To override memory requirements that are specified in the resourceRequirements structure in the job definition, resourceRequirements must be specified in the SubmitJob request, with type set to MEMORY and value set to the new value. For more information, see Can't override job definition resource requirements in the Batch User Guide .
command (list) --
The command to send to the container that overrides the default command from the Docker image or the job definition.
(string) --
instanceType (string) --
The instance type to use for a multi-node parallel job.
Note
This parameter isn't applicable to single-node container jobs or jobs that run on Fargate resources, and shouldn't be provided.
environment (list) --
The environment variables to send to the container. You can add new environment variables, which are added to the container at launch, or you can override the existing environment variables from the Docker image or the job definition.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
A key-value pair object.
name (string) --
The name of the key-value pair. For environment variables, this is the name of the environment variable.
value (string) --
The value of the key-value pair. For environment variables, this is the value of the environment variable.
resourceRequirements (list) --
The type and amount of resources to assign to a container. This overrides the settings in the job definition. The supported resources include GPU , MEMORY , and VCPU .
(dict) --
The type and amount of a resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
value (string) -- [REQUIRED]
The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
type="GPU"
The number of physical GPUs to reserve for the container. Make sure that the number of GPUs reserved for all containers in a job doesn't exceed the number of available GPUs on the compute resource that the job is launched on.
Note
GPUs aren't available for jobs that are running on Fargate resources.
type="MEMORY"
The memory hard limit (in MiB) present to the container. This parameter is supported for jobs that are running on EC2 resources. If your container attempts to exceed the memory specified, the container is terminated. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run . You must specify at least 4 MiB of memory for a job. This is required but can be specified in several places for multi-node parallel (MNP) jobs. It must be specified for each node at least once. This parameter maps to Memory in the Create a container section of the Docker Remote API and the --memory option to docker run .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
For jobs that are running on Fargate resources, then value is the hard limit (in MiB), and must match one of the supported values and the VCPU values must be one of the values supported for that memory value.
value = 512
VCPU = 0.25
value = 1024
VCPU = 0.25 or 0.5
value = 2048
VCPU = 0.25, 0.5, or 1
value = 3072
VCPU = 0.5, or 1
value = 4096
VCPU = 0.5, 1, or 2
value = 5120, 6144, or 7168
VCPU = 1 or 2
value = 8192
VCPU = 1, 2, or 4
value = 9216, 10240, 11264, 12288, 13312, 14336, or 15360
VCPU = 2 or 4
value = 16384
VCPU = 2, 4, or 8
value = 17408, 18432, 19456, 21504, 22528, 23552, 25600, 26624, 27648, 29696, or 30720
VCPU = 4
value = 20480, 24576, or 28672
VCPU = 4 or 8
value = 36864, 45056, 53248, or 61440
VCPU = 8
value = 32768, 40960, 49152, or 57344
VCPU = 8 or 16
value = 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
VCPU = 16
type="VCPU"
The number of vCPUs reserved for the container. This parameter maps to CpuShares in the Create a container section of the Docker Remote API and the --cpu-shares option to docker run . Each vCPU is equivalent to 1,024 CPU shares. For EC2 resources, you must specify at least one vCPU. This is required but can be specified in several places; it must be specified for each node at least once.
The default for the Fargate On-Demand vCPU resource count quota is 6 vCPUs. For more information about Fargate quotas, see Fargate quotas in the Amazon Web Services General Reference .
For jobs that are running on Fargate resources, then value must match one of the supported values and the MEMORY values must be one of the values supported for that VCPU value. The supported values are 0.25, 0.5, 1, 2, 4, 8, and 16
value = 0.25
MEMORY = 512, 1024, or 2048
value = 0.5
MEMORY = 1024, 2048, 3072, or 4096
value = 1
MEMORY = 2048, 3072, 4096, 5120, 6144, 7168, or 8192
value = 2
MEMORY = 4096, 5120, 6144, 7168, 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, or 16384
value = 4
MEMORY = 8192, 9216, 10240, 11264, 12288, 13312, 14336, 15360, 16384, 17408, 18432, 19456, 20480, 21504, 22528, 23552, 24576, 25600, 26624, 27648, 28672, 29696, or 30720
value = 8
MEMORY = 16384, 20480, 24576, 28672, 32768, 36864, 40960, 45056, 49152, 53248, 57344, or 61440
value = 16
MEMORY = 32768, 40960, 49152, 57344, 65536, 73728, 81920, 90112, 98304, 106496, 114688, or 122880
type (string) -- [REQUIRED]
The type of resource to assign to a container. The supported resources include GPU , MEMORY , and VCPU .
dict
The retry strategy to use for failed jobs from this SubmitJob operation. When a retry strategy is specified here, it overrides the retry strategy defined in the job definition.
attempts (integer) --
The number of times to move a job to the RUNNABLE status. You can specify between 1 and 10 attempts. If the value of attempts is greater than one, the job is retried on failure the same number of attempts as the value.
evaluateOnExit (list) --
Array of up to 5 objects that specify the conditions where jobs are retried or failed. If this parameter is specified, then the attempts parameter must also be specified. If none of the listed conditions match, then the job is retried.
(dict) --
Specifies an array of up to 5 conditions to be met, and an action to take (RETRY or EXIT ) if all conditions are met. If none of the EvaluateOnExit conditions in a RetryStrategy match, then the job is retried.
onStatusReason (string) --
Contains a glob pattern to match against the StatusReason returned for a job. The pattern can contain up to 512 characters. It can contain letters, numbers, periods (.), colons (:), and white spaces (including spaces or tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
onReason (string) --
Contains a glob pattern to match against the Reason returned for a job. The pattern can contain up to 512 characters. It can contain letters, numbers, periods (.), colons (:), and white space (including spaces and tabs). It can optionally end with an asterisk (*) so that only the start of the string needs to be an exact match.
onExitCode (string) --
Contains a glob pattern to match against the decimal representation of the ExitCode returned for a job. The pattern can be up to 512 characters long. It can contain only numbers, and can end with an asterisk (*) so that only the start of the string needs to be an exact match.
The string can contain up to 512 characters.
action (string) -- [REQUIRED]
Specifies the action to take if all of the specified conditions (onStatusReason , onReason , and onExitCode ) are met. The values aren't case sensitive.
boolean
Specifies whether to propagate the tags from the job or job definition to the corresponding Amazon ECS task. If no value is specified, the tags aren't propagated. Tags can only be propagated to the tasks during task creation. For tags with the same name, job tags are given priority over job definitions tags. If the total number of combined tags from the job and job definition is over 50, the job is moved to the FAILED state. When specified, this overrides the tag propagation setting in the job definition.
dict
The timeout configuration for this SubmitJob operation. You can specify a timeout duration after which Batch terminates your jobs if they haven't finished. If a job is terminated due to a timeout, it isn't retried. The minimum value for the timeout is 60 seconds. This configuration overrides any timeout configuration specified in the job definition. For array jobs, child jobs have the same timeout configuration as the parent job. For more information, see Job Timeouts in the Amazon Elastic Container Service Developer Guide .
attemptDurationSeconds (integer) --
The job timeout time (in seconds) that's measured from the job attempt's startedAt timestamp. After this time passes, Batch terminates your jobs if they aren't finished. The minimum value for the timeout is 60 seconds.
For array jobs, the timeout applies to the child jobs, not to the parent array job.
For multi-node parallel (MNP) jobs, the timeout applies to the whole job, not to the individual nodes.
dict
The tags that you apply to the job request to help you categorize and organize your resources. Each tag consists of a key and an optional value. For more information, see Tagging Amazon Web Services Resources in Amazon Web Services General Reference .
(string) --
(string) --
dict
An object that can only be specified for jobs that are run on Amazon EKS resources with various properties that override defaults for the job definition.
podProperties (dict) --
The overrides for the Kubernetes pod resources of a job.
containers (list) --
The overrides for the container that's used on the Amazon EKS pod.
(dict) --
Object representing any Kubernetes overrides to a job definition that's used in a SubmitJob API operation.
image (string) --
The override of the Docker image that's used to start the container.
command (list) --
The command to send to the container that overrides the default command from the Docker image or the job definition.
(string) --
args (list) --
The arguments to the entrypoint to send to the container that overrides the default arguments from the Docker image or the job definition. For more information, see CMD in the Dockerfile reference and Define a command an arguments for a pod in the Kubernetes documentation .
(string) --
env (list) --
The environment variables to send to the container. You can add new environment variables, which are added to the container at launch. Or, you can override the existing environment variables from the Docker image or the job definition.
Note
Environment variables cannot start with "AWS_BATCH ". This naming convention is reserved for variables that Batch sets.
(dict) --
An environment variable.
name (string) -- [REQUIRED]
The name of the environment variable.
value (string) --
The value of the environment variable.
resources (dict) --
The type and amount of resources to assign to a container. These override the settings in the job definition. The supported resources include memory , cpu , and nvidia.com/gpu . For more information, see Resource management for pods and containers in the Kubernetes documentation .
limits (dict) --
The type and quantity of the resources to reserve for the container. The values vary based on the name that's specified. Resources can be requested using either the limits or the requests objects.
memory
The memory hard limit (in MiB) for the container, using whole integers, with a "Mi" suffix. If your container attempts to exceed the memory specified, the container is terminated. You must specify at least 4 MiB of memory for a job. memory can be specified in limits , requests , or both. If memory is specified in both places, then the value that's specified in limits must be equal to the value that's specified in requests .
Note
To maximize your resource utilization, provide your jobs with as much memory as possible for the specific instance type that you are using. To learn how, see Memory management in the Batch User Guide .
cpu
The number of CPUs that's reserved for the container. Values must be an even multiple of 0.25 . cpu can be specified in limits , requests , or both. If cpu is specified in both places, then the value that's specified in limits must be at least as large as the value that's specified in requests .
nvidia.com/gpu
The number of GPUs that's reserved for the container. Values must be a whole integer. memory can be specified in limits , requests , or both. If memory is specified in both places, then the value that's specified in limits must be equal to the value that's specified in requests .
(string) --
(string) --
requests (dict) --
The type and quantity of the resources to request for the container. The values vary based on the name that's specified. Resources can be requested by using either the limits or the requests objects.
memory
The memory hard limit (in MiB) for the container, using whole integers, with a "Mi" suffix. If your container attempts to exceed the memory specified, the container is terminated. You must specify at least 4 MiB of memory for a job. memory can be specified in limits , requests , or both. If memory is specified in both, then the value that's specified in limits must be equal to the value that's specified in requests .
Note
If you're trying to maximize your resource utilization by providing your jobs as much memory as possible for a particular instance type, see Memory management in the Batch User Guide .
cpu
The number of CPUs that are reserved for the container. Values must be an even multiple of 0.25 . cpu can be specified in limits , requests , or both. If cpu is specified in both, then the value that's specified in limits must be at least as large as the value that's specified in requests .
nvidia.com/gpu
The number of GPUs that are reserved for the container. Values must be a whole integer. nvidia.com/gpu can be specified in limits , requests , or both. If nvidia.com/gpu is specified in both, then the value that's specified in limits must be equal to the value that's specified in requests .
(string) --
(string) --
metadata (dict) --
labels (dict) --
(string) --
(string) --
dict
Response Syntax
{
'jobArn': 'string',
'jobName': 'string',
'jobId': 'string'
}
Response Structure
(dict) --
jobArn (string) --
The Amazon Resource Name (ARN) for the job.
jobName (string) --
The name of the job.
jobId (string) --
The unique identifier for the job.