API Reference

Generated from the docstrings in the package. Only modules reachable from a live code path are listed; network/, compute/ec2.py, and several utils modules are unreferenced by any current path and are scheduled for removal in v0.8.0 (#90).

Provider

Parsl Ephemeral Provider implementation.

This module implements the main provider class that conforms to the Parsl execution provider interface.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.provider.ComputeType(*values)

Bases: str, Enum

Supported compute resource types.

EC2 = 'ec2'

ECS = 'ecs'

LAMBDA = 'lambda'

class parsl_ephemeral_provider.provider.EphemeralProvider(image_id: str | None = None, instance_type: str = 't3.micro', region: str = 'us-east-1', mode: str = 'standard', min_blocks: int = 0, max_blocks: int = 10, init_blocks: int = 0, nodes_per_block: int = 1, worker_init: str = 'dnf install -y python3.11 python3.11-pip\nln -sf /usr/bin/python3.11 /usr/bin/python3\npip3.11 install --quiet --upgrade parsl\n', vpc_id: str | None = None, subnet_id: str | None = None, security_group_id: str | None = None, key_name: str | None = None, profile_name: str | None = None, endpoint_url: str | None = None, state_store_type: str = StateStoreType.FILE, state_file_path: str = 'ephemeral_aws_state.json', s3_bucket: str | None = None, s3_key: str = 'ephemeral_aws_state.json', s3_create_bucket: bool = False, parameter_store_path: str = '/parsl/ephemeral_aws_state', use_spot: bool = False, spot_max_price: str | None = None, spot_allocation_strategy: str = 'price-capacity-optimized', spot_interruption_handling: bool = False, use_spot_fleet: bool = False, instance_types: List[str] | None = None, spot_max_price_percentage: int | None = None, additional_tags: Dict[str, str] | None = None, auto_shutdown: bool = True, max_idle_time: int = 300, compute_type: str = ComputeType.EC2, bastion_instance_type: str = 't3.micro', idle_timeout: int = 30, preserve_bastion: bool = True, bastion_host_type: str = 'cloudformation', workflow_id: str | None = None, bastion_instance_profile_arn: str | None = None, memory_size: int = 1024, timeout: int = 300, lambda_runtime: str = 'python3.12', ecs_task_cpu: int = 1024, ecs_task_memory: int = 2048, ecs_container_image: str = 'python:3.12-slim', debug: bool = False, use_public_ips: bool = True, custom_ami: bool = False, provider_id: str | None = None, iam_instance_profile_arn: str | None = None, auto_create_instance_profile: bool = False, status_polling_interval: int = 60, waiter_delay: int = 5, waiter_max_attempts: int = 60, warm_pool_size: int = 0, warm_pool_ttl: int = 120, bake_ami: bool = False, baked_ami_id: str | None = None, one_shot: bool = False, cores_per_node: int | None = None, mem_per_node: float | None = None, **kwargs: Any)

Bases: ExecutionProvider, RepresentationMixin

Ephemeral AWS Provider for Parsl.

The Ephemeral AWS Provider allows Parsl to execute tasks on ephemeral AWS resources that are created on-demand and automatically cleaned up when no longer needed.

Parameters

image_idstr, optional

EC2 AMI ID to use for instances. When omitted, the latest Amazon Linux 2023 AMI matching instance_type’s architecture is resolved from AWS’s public SSM parameters. Supply one explicitly for a custom image, or for an instance family AL2023 does not cover (mac*.metal).

instance_typestr, optional

EC2 instance type. Default is ‘t3.micro’. Graviton (arm64) families are supported; the matching arm64 AMI is selected automatically.

regionstr, optional

AWS region. Default is ‘us-east-1’.

modestr, optional

Operating mode (‘standard’, ‘detached’, or ‘serverless’). Default is ‘standard’.

min_blocksint, optional

Minimum number of blocks. Default is 0.

max_blocksint, optional

Maximum number of blocks. Default is 10.

worker_initstr, optional

Initialization script for workers. Default is an empty script.

vpc_idstr

Existing VPC ID to use. Required, and pre-provisioned outside the provider: since #69 the provider creates no VPC, subnet, or security group. Omitting any of the three raises ValueError.

subnet_idstr

Existing subnet ID to use. Required; see vpc_id.

security_group_idstr

Existing security group ID to use. Required; see vpc_id. It is never modified or deleted – a caller-supplied group is not the provider’s to touch (#100).

The one exception to all three is mode="serverless" with compute_type="lambda": functions run in the Lambda-managed VPC, so there is nothing for the caller to pre-provision.

key_namestr, optional

EC2 key pair name for SSH access. If not provided, instances will be created without a key pair.

profile_namestr, optional

AWS profile name to use. If not provided, the default profile will be used.

endpoint_urlstr, optional

Override the endpoint every AWS client uses. Set this for VPC interface endpoints, FIPS endpoints, or to point the provider at an emulator such as substrate. Applies to clients the operating modes and compute managers build from the session too, not just the provider’s own. A caller that needs one service elsewhere can still override it per client.

state_store_typestr, optional

Type of state store to use (‘file’, ‘parameter_store’, or ‘s3’). Default is ‘file’.

state_file_pathstr, optional

Path to state file when using ‘file’ state store. Default is ‘ephemeral_aws_state.json’.

s3_bucketstr, optional

S3 bucket name when using ‘s3’ state store.

s3_keystr, optional

S3 key name when using ‘s3’ state store. Default is ‘ephemeral_aws_state.json’.

s3_create_bucketbool, optional

Whether to create s3_bucket if it does not exist, when using the ‘s3’ state store. Default is False, so a missing bucket is an error rather than something the provider quietly provisions.

parameter_store_pathstr, optional

Parameter Store path when using ‘parameter_store’ state store. Default is ‘/parsl/ephemeral_aws_state’.

use_spotbool, optional

Whether to use spot instances. Default is False.

spot_max_pricestr, optional

Maximum price for spot instances. Default is on-demand price.

spot_allocation_strategystr, optional

Allocation strategy for spot instances, in kebab-case: one of ‘price-capacity-optimized’ (the default, and AWS’s recommendation), ‘capacity-optimized’, ‘capacity-optimized-prioritized’, ‘diversified’, or ‘lowest-price’. Converted to the camelCase spelling Spot Fleet requires at the API boundary.

spot_interruption_handlingbool, optional

Whether to detect spot interruptions. Default is False. When enabled, an interrupted block is reported to Parsl as FAILED rather than COMPLETED, so the executor stops dispatching to it and re-runs the lost tasks under its own retries setting. No checkpointing is performed: a provider never sees task state, so re-running is the only recovery available at this layer. Set retries on your Parsl config to make use of it.

additional_tagsDict[str, str], optional

Additional tags to apply to AWS resources.

auto_shutdownbool, optional

Whether a worker terminates itself once its command finishes. Default is True, which appends shutdown -h now to the worker’s UserData; the instance launches with InstanceInitiatedShutdownBehavior=terminate, so it is terminated rather than stopped with a billed EBS volume. Set False only if you intend to keep instances after their work completes.

max_idle_timeint, optional

Deprecated and ignored; accepted only so existing configurations keep loading. Default is 300.

This never measured idleness. It was compared against a timestamp stamped once at submit, making it wall-clock age since submission, so it terminated any task that ran longer than the limit (#194). A provider cannot compute idleness – that needs per-manager task counts only Parsl’s interchange has. Use Parsl’s own max_idletime, which HighThroughputExecutor.scale_in applies to genuinely idle blocks:

from parsl.config import Config

config = Config(executors=[...], max_idletime=300.0)

compute_typestr, optional

Type of compute resource when using serverless mode (‘ec2’, ‘lambda’, or ‘ecs’). Default is ‘ec2’. 'lambda' and 'ecs' are rejected on the other two modes, which launch EC2 instances; leaving it at 'ec2' on mode="serverless" warns, because ServerlessMode then falls back to its own 'auto' selection rather than honouring the value.

bastion_instance_typestr, optional

Instance type for bastion host when using detached mode. Default is 't3.micro'. mode="detached" only.

idle_timeoutint, optional

Minutes of inactivity before the bastion shuts itself down. Default is 30. mode="detached" only.

preserve_bastionbool, optional

Whether cleanup_infrastructure() leaves the bastion running so a later session can adopt it. Default is True, which means the bastion survives shutdown and keeps billing — set False to have it torn down. mode="detached" only.

bastion_host_typestr, optional

How the bastion is deployed: 'cloudformation' (the default) or 'direct' for a plain RunInstances call. mode="detached" only.

workflow_idstr, optional

Workflow identifier used in bastion state paths and resource tags. Default is a UUID generated by the mode. Supply the same value to reconnect to an existing workflow’s bastion. mode="detached" only.

bastion_instance_profile_arnstr, optional

Instance profile for the bastion to carry. Default is None, which creates one scoped to this workflow and deletes it with the bastion. A profile you supply is never deleted. Only used when bastion_host_type='direct'; the CloudFormation path declares its own inside the stack. mode="detached" only.

memory_sizeint, optional

Memory size in MB for Lambda functions. Default is 1024. The provider-facing name for ServerlessMode’s lambda_memory. mode="serverless" only.

timeoutint, optional

Timeout in seconds for Lambda functions. Default is 300; Lambda’s own ceiling is 900. The provider-facing name for lambda_timeout. mode="serverless" only.

lambda_runtimestr, optional

Runtime identifier for Lambda functions. Default is 'python3.12'. Must be one of the Runtime values allowed by templates/cloudformation/lambda_worker.yml. mode="serverless" only.

ecs_task_cpuint, optional

Fargate CPU units per task. Default is 1024 (1 vCPU). mode="serverless" only.

ecs_task_memoryint, optional

Fargate memory per task in MB. Default is 2048. Must be a combination Fargate accepts for the chosen ecs_task_cpu. mode="serverless" only.

ecs_container_imagestr, optional

Container image for Fargate tasks. Default is 'python:3.12-slim'. Set this to your own image to run a workload with its own dependencies — without it every task runs the stock image and can only use the standard library. mode="serverless" only.

debugbool, optional

Whether to enable debug logging. Default is False.

use_public_ipsbool, optional

Whether to assign public IPs to instances. Default is True.

custom_amibool, optional

Whether image_id refers to a custom AMI. Default is False.

provider_idstr, optional

Provider ID for distinguishing between multiple providers. Default is a UUID.

iam_instance_profile_arnstr, optional

ARN of an existing IAM instance profile to attach to EC2 instances. Required for SSM connectivity when using Session Manager tunneling.

auto_create_instance_profilebool, optional

When True, automatically create an IAM role and instance profile with AmazonSSMManagedInstanceCore permissions if one does not already exist. Default is False.

status_polling_intervalint, optional

Interval in seconds between status polls. Default is 60.

waiter_delayint, optional

Seconds between waiter attempts when polling for resource state changes. Default is 5.

waiter_max_attemptsint, optional

Maximum number of waiter attempts before raising an error. Default is 60 (5 minutes at the default delay).

warm_pool_sizeint, optional

Maximum number of instances to keep warm after their job finishes, ready for immediate reuse without re-running worker_init. Requires auto_create_instance_profile=True or iam_instance_profile_arn (SSM SendCommand needs an IAM role). Default is 0 (disabled). mode="standard" only.

This costs money while idle. A warm instance is left Running, not Stopped, so it bills at the full instance rate for up to warm_pool_ttl seconds after its job finishes — warm_pool_size instances × warm_pool_ttl seconds of instance time per idle period, whether or not another job ever arrives to use them. Capped at MAX_WARM_POOL_SIZE (20). Instances must stay Running because dispatch is SSM SendCommand and a Stopped instance runs no SSM agent; issue #130 tracks moving to a pull model so the pool can be Stopped instead.

warm_pool_ttlint, optional

Seconds a warm running instance stays alive before being terminated. Default is 120 (2 minutes); this was 600 in v0.6.0 and was reduced because the instance bills for the whole window. mode="standard" only.

bake_amibool, optional

When True, run worker_init on a builder instance during initialize(), snapshot it into a custom AMI, and use that AMI for all subsequent instance launches. Eliminates the per-boot install overhead for new instances. Default is False. mode="standard" only.

baked_ami_idstr, optional

Pre-existing baked AMI ID to use instead of baking a new one. When supplied, initialize() skips the baking step and uses this AMI directly for all instance launches. mode="standard" only.

one_shotbool, optional

When True, each instance runs a single command over SSM and then terminates, so the command’s exit code determines the job status. Default is False. mode="standard" only.

Raises

ProviderConfigurationError

If warm_pool_size, warm_pool_ttl, bake_ami, baked_ami_id, or one_shot is set on any mode other than "standard" — no other mode implements them.

If idle_timeout, preserve_bastion, bastion_host_type, workflow_id, or bastion_instance_profile_arn is set on any mode other than "detached", or lambda_runtime, ecs_task_cpu, ecs_task_memory, or ecs_container_image on any mode other than "serverless" — each is forwarded from one mode’s branch only, so it would silently have no effect.

Initialize the Ephemeral AWS Provider.

__init__(image_id: str | None = None, instance_type: str = 't3.micro', region: str = 'us-east-1', mode: str = 'standard', min_blocks: int = 0, max_blocks: int = 10, init_blocks: int = 0, nodes_per_block: int = 1, worker_init: str = 'dnf install -y python3.11 python3.11-pip\nln -sf /usr/bin/python3.11 /usr/bin/python3\npip3.11 install --quiet --upgrade parsl\n', vpc_id: str | None = None, subnet_id: str | None = None, security_group_id: str | None = None, key_name: str | None = None, profile_name: str | None = None, endpoint_url: str | None = None, state_store_type: str = StateStoreType.FILE, state_file_path: str = 'ephemeral_aws_state.json', s3_bucket: str | None = None, s3_key: str = 'ephemeral_aws_state.json', s3_create_bucket: bool = False, parameter_store_path: str = '/parsl/ephemeral_aws_state', use_spot: bool = False, spot_max_price: str | None = None, spot_allocation_strategy: str = 'price-capacity-optimized', spot_interruption_handling: bool = False, use_spot_fleet: bool = False, instance_types: List[str] | None = None, spot_max_price_percentage: int | None = None, additional_tags: Dict[str, str] | None = None, auto_shutdown: bool = True, max_idle_time: int = 300, compute_type: str = ComputeType.EC2, bastion_instance_type: str = 't3.micro', idle_timeout: int = 30, preserve_bastion: bool = True, bastion_host_type: str = 'cloudformation', workflow_id: str | None = None, bastion_instance_profile_arn: str | None = None, memory_size: int = 1024, timeout: int = 300, lambda_runtime: str = 'python3.12', ecs_task_cpu: int = 1024, ecs_task_memory: int = 2048, ecs_container_image: str = 'python:3.12-slim', debug: bool = False, use_public_ips: bool = True, custom_ami: bool = False, provider_id: str | None = None, iam_instance_profile_arn: str | None = None, auto_create_instance_profile: bool = False, status_polling_interval: int = 60, waiter_delay: int = 5, waiter_max_attempts: int = 60, warm_pool_size: int = 0, warm_pool_ttl: int = 120, bake_ami: bool = False, baked_ami_id: str | None = None, one_shot: bool = False, cores_per_node: int | None = None, mem_per_node: float | None = None, **kwargs: Any) → None

Initialize the Ephemeral AWS Provider.

__repr__() → str

Return string representation of the provider.

Returns

str

String representation.

cancel(job_ids: Sequence[object]) → List[bool]

Cancel specified jobs.

Parameters

job_idsSequence[object]

Job identifiers to cancel. Typed as object to match Parsl’s ExecutionProvider; an ID this provider never issued reports False rather than raising.

Returns

List[bool]

True for each job_id where cancellation was accepted, False otherwise.

cleanup_all() → None

Clean up all resources created by this provider.

image_id: str | None

job_map: Dict[str, Dict[str, Any]]

property label: str

Return the label for the provider.

Returns

str

Provider label.

list_resources() → Dict[str, List[Dict[str, Any]]]

List all resources created by this provider.

Returns

Dict[str, List[Dict[str, Any]]]

Dictionary of resource types and their details.

resources: Dict[object, Any]

scale_in(blocks: int) → List[str]

Scale in the number of blocks by the specified amount.

Parameters

blocksint

Number of blocks to scale in by.

Returns

List[str]

List of job IDs that were terminated.

scale_out(blocks: int) → List[str]

Scale out resources by the specified number of blocks.

Parameters

blocksint

Number of blocks to scale out by.

Returns

List[str]

List of job IDs for the new resources.

script_dir: str | None

shutdown() → None

Shutdown the provider and cleanup all resources.

status(job_ids: Sequence[object]) → List[JobStatus]

Get the status of a list of jobs.

Parameters

job_idsSequence[object]

Job identifiers as returned by submit(). Typed as object to match Parsl’s ExecutionProvider, which treats job IDs as opaque; this provider issues strings, so anything else resolves to JobState.UNKNOWN rather than raising.

Returns

List[JobStatus]

List of JobStatus objects corresponding to each job_id.

property status_polling_interval: int

Return the status polling interval for the provider.

Returns

int

Polling interval in seconds.

submit(command: str, tasks_per_node: int, job_name: str = 'parsl.auto') → str

Submit a job to execute the specified command.

Parameters

commandstr

Command to execute.

tasks_per_nodeint

Number of tasks to run per node.

job_namestr

Human-friendly name for the job request. Defaults to Parsl’s "parsl.auto" sentinel, which is replaced with a unique generated name.

Returns

str

Job ID for tracking status.

class parsl_ephemeral_provider.provider.OperatingModeType(*values)

Bases: str, Enum

Supported operating modes for the provider.

DETACHED = 'detached'

SERVERLESS = 'serverless'

STANDARD = 'standard'

class parsl_ephemeral_provider.provider.StateStoreType(*values)

Bases: str, Enum

Supported state persistence options.

FILE = 'file'

PARAMETER_STORE = 'parameter_store'

S3 = 's3'

Parsl Ephemeral Compute Provider for Globus Compute.

An independent Parsl provider for running ephemeral compute on AWS through Globus Compute. AWS stays named here because it is the platform the compute actually runs on – this is a subclass of the AWS provider, not an alternative to it – and because a description is exactly where the AWS Trademark Guidelines permit the mark (s13, plain-text factual reference); s7 keeps it out of the identifiers.

Exposes EphemeralComputeProvider, a thin subclass of EphemeralProvider that:

• Carries endpoint_id and container_image metadata for Globus Compute.

• Provides generate_endpoint_config(path) which writes a Globus Compute endpoint configuration the globus-compute-endpoint daemon can load.

Usage:

from parsl_ephemeral_provider import EphemeralComputeProvider

provider = EphemeralComputeProvider(
    endpoint_id="<your-globus-endpoint-uuid>",
    region="us-east-1",
    instance_type="t3.medium",
    mode="standard",
    vpc_id="vpc-...",
    subnet_id="subnet-...",
    security_group_id="sg-...",
    use_spot=True,
    auto_create_instance_profile=True,
    display_name="My Ephemeral AWS Endpoint",
)
provider.generate_endpoint_config("~/.globus_compute/my_aws_endpoint")

The shape of a startable endpoint

Every endpoint globus-compute-endpoint 4.15.0 can start is a manager endpoint (MEP). The classification is made by one key: load_config_yaml() pops engine and picks ManagerEndpointConfig when it is absent, UserEndpointConfig when it is present. start then refuses anything that is not a ManagerEndpointConfig (cli.py:899), and the only other entry point – _start-user-endpoint – reads its config from stdin and is invoked solely by a running manager. So a config.yaml carrying a top-level engine: block cannot be started at all: that was #196.

generate_endpoint_config() therefore writes the manager/template pair upstream’s own configure produces:

config.yaml

Manager configuration. display_name and nothing else that matters – upstream’s packaged default_config.yaml is the single line display_name: null.

user_config_template.yaml.j2

The engine: block, including the provider: sub-block that names this class. The manager renders this per user endpoint.

How Globus Compute finds this provider

Globus Compute resolves the provider: type: key by plain attribute lookup on the parsl.providers module – getattr(parsl.providers, type_name, None), raising if the result is None (globus_compute_endpoint/endpoint/config/dispatch.py). Two consequences, both verified against 4.15.0:

1. A dotted path can never resolve, because getattr does not walk dots. The type key must therefore be the bare class name.

2. The bare name only resolves if something has already assigned the class onto parsl.providers. Importing this package does that (see _register_with_parsl_providers()), but nothing in the endpoint’s own startup has a reason to import it.

The template is rendered and loaded in a different interpreter from the manager: EndpointManager forks and os.execvpe``s ``globus-compute-endpoint _start-user-endpoint <name>, and that child calls load_config_yaml() on the rendered string handed to it on stdin. So the config.py shim this package used to write (#87) cannot help – get_config() is never reached in the child, and the import has to happen before its first line of user code.

The seam that does reach it is user_environment.yaml, which the manager reads and merges into the child’s environment immediately before execvpe (endpoint_manager.py:1069). generate_endpoint_config() writes a PYTHONPATH there pointing at a _bootstrap/ directory holding a sitecustomize.py whose body is import parsl_ephemeral_provider – so the interpreter registers the class during site initialisation, before the config is parsed. Verified in a genuinely fresh interpreter: with the PYTHONPATH the bare name resolves, without it getattr returns None.

Dotted-path support upstream (#133) would make the bootstrap unnecessary. Until then this is what makes a generated endpoint start.

One platform caveat: start requires pyprctl, which is Linux-only, so on macOS it exits “multi-user endpoints are not supported on this system” before reading any configuration. Generation works anywhere; running an endpoint needs Linux, and that is true of every 4.15.0 endpoint, not just these.

Minimum IAM permissions

EphemeralComputeProvider.minimum_iam_policy() returns these as a policy document. The lists are derived from the AWS API calls the package actually makes on the mode="standard" path, which is what a generated endpoint config uses; AWS may require further implicit permissions.

EC2 (always required)

ec2:RunInstances, ec2:TerminateInstances, ec2:DescribeInstances, ec2:DescribeInstanceTypes, ec2:CreateTags, ec2:DescribeTags, ec2:DescribeImages, ec2:CreateImage, ec2:DeregisterImage, ec2:DeleteSnapshot, ec2:DescribeVpcs, ec2:DescribeSubnets, ec2:DescribeSecurityGroups, ec2:CreateLaunchTemplate, ec2:CreateLaunchTemplateVersion, ec2:DeleteLaunchTemplate, ec2:CreateFleet, ec2:DescribeFleets, ec2:DeleteFleets, ec2:RequestSpotInstances, ec2:DescribeSpotInstanceRequests, ec2:DescribeSpotPriceHistory

The network resources are read-only: vpc_id, subnet_id and security_group_id have been caller-supplied since v0.7.0, so no create/delete grant is needed for them.

SSM (required)

ssm:GetParameter (resolves the current Amazon Linux 2023 AMI), ssm:SendCommand, ssm:GetCommandInvocation, ssm:DescribeInstanceInformation (warm-pool and one-shot dispatch), ssm:PutParameter, ssm:DeleteParameter, ssm:DeleteParameters (state_store_type="parameter_store")

No Session Manager grants. ssm:StartSession, TerminateSession, ResumeSession, DescribeSessions and GetConnectionStatus were listed here for “Session Manager tunnels”, but no such transport exists in this package and nothing calls them (#195).

STS (always required)

sts:GetCallerIdentity – create_session() verifies every session with it, so this is the first AWS call the provider makes.

EventBridge + SQS (required when use_spot and spot_interruption_handling)

events:PutRule, events:PutTargets, events:RemoveTargets, events:DeleteRule, events:TagResource, sqs:CreateQueue, sqs:GetQueueAttributes, sqs:SetQueueAttributes, sqs:ReceiveMessage, sqs:DeleteMessage, sqs:DeleteQueue

IAM (required when auto_create_instance_profile=True)

iam:CreateRole, iam:GetRole, iam:AttachRolePolicy, iam:CreateInstanceProfile, iam:GetInstanceProfile, iam:AddRoleToInstanceProfile, iam:PassRole, iam:RemoveRoleFromInstanceProfile, iam:DeleteInstanceProfile, iam:ListAttachedRolePolicies, iam:DetachRolePolicy, iam:ListRolePolicies, iam:DeleteRolePolicy, iam:DeleteRole

The teardown half is required, not optional: cleanup_infrastructure() deletes the pair it created (#132), and cleanup logs rather than raises, so a missing delete grant leaks a standing privileged principal silently (#195).

ECR (only when container_image references an ECR repository)

ecr:GetAuthorizationToken, ecr:BatchGetImage, ecr:GetDownloadUrlForLayer, ecr:BatchCheckLayerAvailability

Not covered by minimum_iam_policy(): state_store_type="s3" (needs S3 object access), mode="detached" (CloudFormation stack management), and mode="serverless" (Lambda, ECS, CloudWatch Logs, and IAM role lifecycle). The Parameter Store backend is covered now – it was listed here as uncovered while the policy also omitted the actions, so the omission read as deliberate (#195).

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.globus_compute.EphemeralComputeProvider(endpoint_id: str | None = None, container_image: str | None = None, display_name: str = 'Ephemeral AWS Endpoint', encrypted: bool = False, **kwargs: Any)

Bases: EphemeralProvider

Globus Compute-aware wrapper around EphemeralProvider.

Extends EphemeralProvider with Globus Compute endpoint metadata and a helper that generates a ready-to-start endpoint directory for the globus-compute-endpoint daemon.

All EphemeralProvider constructor parameters are accepted as-is (forwarded via **kwargs).

Parameters

endpoint_idstr, optional

Globus Compute endpoint UUID. Optional, and it is not a configuration key – Globus Compute’s BaseConfig rejects endpoint_id, and the UUID lives in endpoint.json written during registration. When set, the generated config.yaml records it as a comment together with the --endpoint-uuid invocation that adopts it; when unset, start registers the endpoint and assigns one.

container_imagestr, optional

Container image URI to run Parsl workers inside a container. Examples: "python:3.11-slim", "123456789.dkr.ecr.us-east-1.amazonaws.com/my-image:latest". When set, the generated user_config_template.yaml.j2 includes container_type: docker and container_uri: <image> under the engine block.

display_namestr, optional

Human-readable name for the Globus Compute endpoint. Default is "Ephemeral AWS Endpoint".

encryptedbool, optional

Whether the engine encrypts worker traffic with CurveZMQ. Default is False, and that default is deliberate: HighThroughputExecutor generates the certificates under its own run_dir on the endpoint host and passes that path to workers as --cert_dir, so an EC2 worker is handed a directory that does not exist on it and dies with FileNotFoundError. Until certificate distribution is implemented (#62), True cannot work for remote workers and rely on VPC isolation instead. This was hardcoded true before #138, which made every generated config unusable.

High-Assurance endpoints are the exception – assert_ha_compliant() rejects encrypted=False, so those need #62 resolved rather than this default.

**kwargs

All keyword arguments accepted by EphemeralProvider.

Initialize the Ephemeral AWS Provider.

generate_endpoint_config(path: str) → str

Write a startable Globus Compute endpoint configuration to path.

Creates the directory at path if it does not exist, then writes four files into it:

config.yaml

Manager configuration: display_name, and nothing else. Deliberately thin – upstream’s own packaged default is the single line display_name: null.

user_config_template.yaml.j2

The engine: block and its provider: sub-block, which is where all the AWS configuration lives. The file to edit.

user_environment.yaml

A PYTHONPATH pointing at _bootstrap/.

_bootstrap/sitecustomize.py

import parsl_ephemeral_provider, which registers EphemeralComputeProvider on parsl.providers.

The split between the first two is not cosmetic: an engine: key in config.yaml makes load_config_yaml() return a UserEndpointConfig, and start rejects anything that is not a ManagerEndpointConfig – so the previous single-file output could never be started (#196). The last two exist because the process that loads the template is a fresh interpreter that never imports this package; see the module docstring.

Returns the absolute path to user_config_template.yaml.j2 – the file a caller would want to read or edit, and the one holding everything this class configures.

The result is ready for the globus-compute-endpoint daemon:

globus-compute-endpoint start my_aws_endpoint

Parameters

pathstr

Path to the Globus Compute endpoint directory (e.g. "~/.globus_compute/my_aws_endpoint").

Returns

str

Absolute path to the written user_config_template.yaml.j2.

label = 'globus_compute_aws'

static minimum_iam_policy(include_ecr: bool = False) → Dict[str, Any]

Return the minimum IAM policy document as a Python dict.

The returned dict can be serialised to JSON and attached to the IAM principal that runs the provider – the user, role, or endpoint host that calls submit(). It is not the instance role: workers need only AmazonSSMManagedInstanceCore, which auto_create_instance_profile=True attaches for them.

Scope: the mode="standard" path, which is what a generated endpoint config uses, with either file-backed or Parameter Store state. Actions were derived from the package’s actual API calls, so the set is narrower than it was before v0.7.0 – network resources are caller-supplied since #69, so no VPC/subnet/security-group/NAT/gateway create or delete grant appears, and Spot Fleet was replaced by EC2 Fleet in #86. See the module docstring for what is deliberately not covered (the S3 state backend, and detached and serverless modes).

Every action here has a call site in the package, and the teardown actions matter as much as the create ones: cleanup logs rather than raises, so a missing delete permission leaks resources silently. That is what #195 found – this method granted iam:CreateRole and iam:CreateInstanceProfile with no corresponding deletes, so a user on this policy reproduced the very leak #132 had just fixed.

Parameters

include_ecrbool

When True, include ECR permissions required to pull images from a private ECR repository (needed when container_image references an ECR URI).

Returns

dict

IAM policy document compatible with json.dumps().

Operating modes

Base operating mode interface for the EphemeralProvider.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.modes.base.OperatingMode(provider_id: str, session: Session, state_store: StateStore, image_id: str | None = None, instance_type: str = 't3.micro', worker_init: str = '', vpc_id: str | None = None, subnet_id: str | None = None, security_group_id: str | None = None, key_name: str | None = None, use_spot: bool = False, spot_max_price: str | None = None, spot_allocation_strategy: str = 'price-capacity-optimized', spot_interruption_handling: bool = False, additional_tags: Dict[str, str] | None = None, auto_shutdown: bool = True, max_idle_time: int = 300, use_public_ips: bool = True, custom_ami: bool = False, debug: bool = False, region: str | None = None, require_network_resources: bool = True, **kwargs: Any)

Bases: ABC

Abstract base class for provider operating modes.

An operating mode defines how the provider interacts with AWS resources to execute jobs. Different modes have different trade-offs in terms of cost, performance, and capabilities.

Attributes

provider_idstr

Unique identifier for the provider instance

sessionboto3.Session

AWS session for API calls

state_storeStateStore

Store for persisting state

image_idOptional[str]

EC2 AMI ID to use for instances

instance_typestr

EC2 instance type for compute resources

worker_initstr

Script to execute during worker initialization

vpc_idOptional[str]

Existing VPC ID to use

subnet_idOptional[str]

Existing subnet ID to use

security_group_idOptional[str]

Existing security group ID to use

key_nameOptional[str]

EC2 key pair name for SSH access

use_spotbool

Whether to use spot instances

spot_max_priceOptional[str]

Maximum price for spot instances

spot_allocation_strategystr

Allocation strategy for spot instances

spot_interruption_handlingbool

Whether to detect spot interruptions and mark the affected block failed

additional_tagsDict[str, str]

Tags to apply to created resources

auto_shutdownbool

Whether a worker terminates itself once its command finishes

max_idle_timeint

Deprecated and ignored; retained so older state files still load (#194)

use_public_ipsbool

Whether to assign public IPs to instances

custom_amibool

Whether image_id refers to a custom AMI

debugbool

Whether to enable debug logging

Initialize the operating mode.

Parameters

provider_idstr

Unique identifier for the provider instance

sessionboto3.Session

AWS session for API calls

state_storeStateStore

Store for persisting state

image_idOptional[str], optional

EC2 AMI ID to use for instances, by default None

instance_typestr, optional

EC2 instance type for compute resources, by default “t3.micro”

worker_initstr, optional

Script to execute during worker initialization, by default “”

vpc_idOptional[str], optional

Existing VPC ID to use, by default None

subnet_idOptional[str], optional

Existing subnet ID to use, by default None

security_group_idOptional[str], optional

Existing security group ID to use, by default None

key_nameOptional[str], optional

EC2 key pair name for SSH access, by default None

use_spotbool, optional

Whether to use spot instances, by default False

spot_max_priceOptional[str], optional

Maximum price for spot instances, by default None

spot_allocation_strategystr, optional

Allocation strategy for spot instances, in kebab-case, by default “price-capacity-optimized”

spot_interruption_handlingbool, optional

Whether to detect spot interruptions, by default False. Detection marks the affected block STATUS_INTERRUPTED, which the provider reports to Parsl as FAILED so it re-runs the lost tasks.

additional_tagsOptional[Dict[str, str]], optional

Tags to apply to created resources, by default None

auto_shutdownbool, optional

Whether a worker terminates itself once its command finishes, by default True

max_idle_timeint, optional

Deprecated and ignored, by default 300. Nothing reads it; it is kept so state files written by earlier versions still load. Use Parsl’s own max_idletime to reclaim idle blocks (#194).

use_public_ipsbool, optional

Whether to assign public IPs to instances, by default True

custom_amibool, optional

Whether image_id refers to a custom AMI, by default False

debugbool, optional

Whether to enable debug logging, by default False

require_network_resourcesbool, optional

Whether vpc_id, subnet_id, and security_group_id are mandatory, by default True. Subclasses whose compute backend supplies its own networking (e.g. Lambda-only serverless mode) pass False.

__init__(provider_id: str, session: Session, state_store: StateStore, image_id: str | None = None, instance_type: str = 't3.micro', worker_init: str = '', vpc_id: str | None = None, subnet_id: str | None = None, security_group_id: str | None = None, key_name: str | None = None, use_spot: bool = False, spot_max_price: str | None = None, spot_allocation_strategy: str = 'price-capacity-optimized', spot_interruption_handling: bool = False, additional_tags: Dict[str, str] | None = None, auto_shutdown: bool = True, max_idle_time: int = 300, use_public_ips: bool = True, custom_ami: bool = False, debug: bool = False, region: str | None = None, require_network_resources: bool = True, **kwargs: Any) → None

Initialize the operating mode.

Parameters

provider_idstr

Unique identifier for the provider instance

sessionboto3.Session

AWS session for API calls

state_storeStateStore

Store for persisting state

image_idOptional[str], optional

EC2 AMI ID to use for instances, by default None

instance_typestr, optional

EC2 instance type for compute resources, by default “t3.micro”

worker_initstr, optional

Script to execute during worker initialization, by default “”

vpc_idOptional[str], optional

Existing VPC ID to use, by default None

subnet_idOptional[str], optional

Existing subnet ID to use, by default None

security_group_idOptional[str], optional

Existing security group ID to use, by default None

key_nameOptional[str], optional

EC2 key pair name for SSH access, by default None

use_spotbool, optional

Whether to use spot instances, by default False

spot_max_priceOptional[str], optional

Maximum price for spot instances, by default None

spot_allocation_strategystr, optional

Allocation strategy for spot instances, in kebab-case, by default “price-capacity-optimized”

spot_interruption_handlingbool, optional

Whether to detect spot interruptions, by default False. Detection marks the affected block STATUS_INTERRUPTED, which the provider reports to Parsl as FAILED so it re-runs the lost tasks.

additional_tagsOptional[Dict[str, str]], optional

Tags to apply to created resources, by default None

auto_shutdownbool, optional

Whether a worker terminates itself once its command finishes, by default True

max_idle_timeint, optional

Deprecated and ignored, by default 300. Nothing reads it; it is kept so state files written by earlier versions still load. Use Parsl’s own max_idletime to reclaim idle blocks (#194).

use_public_ipsbool, optional

Whether to assign public IPs to instances, by default True

custom_amibool, optional

Whether image_id refers to a custom AMI, by default False

debugbool, optional

Whether to enable debug logging, by default False

require_network_resourcesbool, optional

Whether vpc_id, subnet_id, and security_group_id are mandatory, by default True. Subclasses whose compute backend supplies its own networking (e.g. Lambda-only serverless mode) pass False.

abstractmethod cancel_jobs(resource_ids: List[str]) → Dict[str, str]

Cancel jobs.

Parameters

resource_idsList[str]

List of resource IDs to cancel

Returns

Dict[str, str]

Dictionary mapping resource IDs to status strings

abstractmethod cleanup_all() → None

Clean up all resources created by this mode.

abstractmethod cleanup_infrastructure() → None

Clean up infrastructure created by this mode.

This should clean up any VPC, subnets, security groups, etc. created by the mode.

abstractmethod cleanup_resources(resource_ids: List[str]) → None

Clean up resources.

Parameters

resource_idsList[str]

List of resource IDs to clean up

delete_state() → None

Delete the state stored under the mode’s own state key.

Called on provider shutdown. The provider deletes its own key separately; leaving either behind strands a document that describes resources which no longer exist.

ensure_initialized() → None

Ensure the mode is initialized.

Raises

OperatingModeError

If initialization fails

abstractmethod get_job_status(resource_ids: List[str]) → Dict[str, str]

Get the status of jobs.

Parameters

resource_idsList[str]

List of resource IDs to check

Returns

Dict[str, str]

Dictionary mapping resource IDs to status strings

handle_fleet_interruption(fleet_id: str, instance_ids: List[str], event: Dict[str, Any]) → None

Mark a fleet’s block, and each warned instance, interrupted.

Both are marked: the block is what Parsl holds a job ID for, while the instances are what the monitor names, and either may be the tracked resource depending on the launch path.

A fleet ID is almost never a resource key. resources is keyed by block ID in StandardMode and by serverless-<job_id> in the other two, with the fleet recorded as a fleet_request_id field on the record – so a direct resources[fleet_id] lookup misses every time and the block would keep reporting healthy while AWS took its capacity away. The field is searched instead.

Parameters

fleet_idstr

The fleet AWS has warned about.

instance_idsList[str]

Instances within the fleet being reclaimed.

eventDict[str, Any]

The interruption event, logged for diagnosis.

handle_instance_interruption(instance_id: str, event: Dict[str, Any]) → None

Mark instance_id interrupted so the block stops being dispatched to.

Registered with SpotInterruptionMonitor as the per-instance callback and invoked on the two-minute reclaim warning, roughly 15 s after AWS issues it.

Marking the resource is the whole response, and it is the useful one: get_job_status reports STATUS_INTERRUPTED, the provider maps that to JobState.FAILED, and Parsl stops dispatching to the block and re-runs its tasks under the executor’s own retries. Nothing here needs S3.

Without this the interruption was invisible rather than merely unhandled: the instance moves to shutting-down, which EC2_STATUS_MAPPING renders COMPLETED, so a reclaimed block reported success and its tasks were dropped silently (#137).

Parameters

instance_idstr

The instance AWS has warned about.

eventDict[str, Any]

The interruption event, logged for diagnosis.

abstractmethod initialize() → None

Initialize mode-specific resources.

This method should create any resources needed for the mode to operate, such as VPC, subnets, security groups, etc.

Raises

ResourceCreationError

If resource creation fails

abstractmethod list_resources() → Dict[str, List[Dict[str, Any]]]

List all resources created by this mode.

Returns

Dict[str, List[Dict[str, Any]]]

Dictionary of resource types and their details

load_state() → bool

Load state from the mode’s own state key.

Returns

bool

True if state was loaded successfully, False otherwise

save_state() → None

Save the current state under the mode’s own state key.

The provider writes STATE_KEY_PROVIDER separately; see EphemeralProvider._save_state.

abstractmethod submit_job(job_id: str, command: str, tasks_per_node: int, job_name: str | None = None) → str

Submit a job for execution.

Parameters

job_idstr

Unique identifier for the job

commandstr

Command to execute

tasks_per_nodeint

Number of tasks to run per node

job_nameOptional[str], optional

Human-readable name for the job, by default None

Returns

str

Resource ID for tracking the job

Raises

OperatingModeError

If job submission fails

Standard operating mode for the EphemeralProvider.

The standard mode uses EC2 instances for computation with direct communication between the client and worker nodes.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.modes.standard.StandardMode(provider_id: str, session: Session, state_store: Any, image_id: str | None = None, instance_type: str = 't3.micro', worker_init: str = '', vpc_id: str | None = None, subnet_id: str | None = None, security_group_id: str | None = None, key_name: str | None = None, use_spot: bool = False, spot_max_price: str | None = None, spot_allocation_strategy: str = 'price-capacity-optimized', additional_tags: Dict[str, str] | None = None, auto_shutdown: bool = True, max_idle_time: int = 300, use_public_ips: bool = True, custom_ami: bool = False, debug: bool = False, use_spot_fleet: bool = False, instance_types: List[str] | None = None, nodes_per_block: int = 1, spot_max_price_percentage: int | None = None, warm_pool_size: int = 0, warm_pool_ttl: int = 600, iam_instance_profile_arn: str | None = None, auto_create_instance_profile: bool = False, bake_ami: bool = False, baked_ami_id: str | None = None, one_shot: bool = False, **kwargs: Any)

Bases: OperatingMode

Standard operating mode implementation.

In standard mode, EC2 instances are created for computation with direct communication between the client and worker nodes.

This mode supports regular EC2 instances, spot instances, and spot fleet requests for more reliable and cost-effective computation.

Initialize the standard mode.

Parameters

provider_idstr

Unique identifier for the provider instance

sessionboto3.Session

AWS session for API calls

state_storeAny

Store for persisting state

image_idOptional[str], optional

EC2 AMI ID to use for instances, by default None

instance_typestr, optional

EC2 instance type for compute resources, by default “t3.micro”

worker_initstr, optional

Script to execute during worker initialization, by default “”

vpc_idOptional[str], optional

Existing VPC ID to use, by default None

subnet_idOptional[str], optional

Existing subnet ID to use, by default None

security_group_idOptional[str], optional

Existing security group ID to use, by default None

key_nameOptional[str], optional

EC2 key pair name for SSH access, by default None

use_spotbool, optional

Whether to use spot instances, by default False

spot_max_priceOptional[str], optional

Maximum price for spot instances, by default None

spot_allocation_strategystr, optional

Allocation strategy for spot instances, in kebab-case, by default “price-capacity-optimized”

additional_tagsOptional[Dict[str, str]], optional

Tags to apply to created resources, by default None

auto_shutdownbool, optional

Whether a worker terminates itself once its command finishes, by default True. Appends shutdown -h now to the worker’s UserData.

max_idle_timeint, optional

Deprecated and ignored, by default 300. Use Parsl’s own max_idletime to reclaim idle blocks (#194).

use_public_ipsbool, optional

Whether to assign public IPs to instances, by default True

custom_amibool, optional

Whether image_id refers to a custom AMI, by default False

debugbool, optional

Whether to enable debug logging, by default False

use_spot_fleetbool, optional

Whether to use Spot Fleet for spot instances, by default False

instance_typesOptional[List[str]], optional

List of instance types to use with Spot Fleet, by default None

nodes_per_blockint, optional

Number of nodes per block, by default 1

spot_max_price_percentageOptional[int], optional

Maximum spot price as a percentage of on-demand price, by default None

warm_pool_sizeint, optional

Number of idle instances to keep for reuse, by default 0 (disabled)

warm_pool_ttlint, optional

Seconds a warm instance is kept before termination, by default 600

iam_instance_profile_arnOptional[str], optional

Instance profile ARN granting SSM access, by default None

auto_create_instance_profilebool, optional

Whether to create an instance profile with the AmazonSSMManagedInstanceCore policy when iam_instance_profile_arn is not supplied, by default False. SSM SendCommand dispatch cannot work without one of the two.

bake_amibool, optional

Whether to pre-install worker_init into a custom AMI, by default False

baked_ami_idOptional[str], optional

Pre-baked AMI to use instead of baking one, by default None

one_shotbool, optional

Whether to dispatch a single command per instance and terminate, by default False

__init__(provider_id: str, session: Session, state_store: Any, image_id: str | None = None, instance_type: str = 't3.micro', worker_init: str = '', vpc_id: str | None = None, subnet_id: str | None = None, security_group_id: str | None = None, key_name: str | None = None, use_spot: bool = False, spot_max_price: str | None = None, spot_allocation_strategy: str = 'price-capacity-optimized', additional_tags: Dict[str, str] | None = None, auto_shutdown: bool = True, max_idle_time: int = 300, use_public_ips: bool = True, custom_ami: bool = False, debug: bool = False, use_spot_fleet: bool = False, instance_types: List[str] | None = None, nodes_per_block: int = 1, spot_max_price_percentage: int | None = None, warm_pool_size: int = 0, warm_pool_ttl: int = 600, iam_instance_profile_arn: str | None = None, auto_create_instance_profile: bool = False, bake_ami: bool = False, baked_ami_id: str | None = None, one_shot: bool = False, **kwargs: Any) → None

Initialize the standard mode.

Parameters

provider_idstr

Unique identifier for the provider instance

sessionboto3.Session

AWS session for API calls

state_storeAny

Store for persisting state

image_idOptional[str], optional

EC2 AMI ID to use for instances, by default None

instance_typestr, optional

EC2 instance type for compute resources, by default “t3.micro”

worker_initstr, optional

Script to execute during worker initialization, by default “”

vpc_idOptional[str], optional

Existing VPC ID to use, by default None

subnet_idOptional[str], optional

Existing subnet ID to use, by default None

security_group_idOptional[str], optional

Existing security group ID to use, by default None

key_nameOptional[str], optional

EC2 key pair name for SSH access, by default None

use_spotbool, optional

Whether to use spot instances, by default False

spot_max_priceOptional[str], optional

Maximum price for spot instances, by default None

spot_allocation_strategystr, optional

Allocation strategy for spot instances, in kebab-case, by default “price-capacity-optimized”

additional_tagsOptional[Dict[str, str]], optional

Tags to apply to created resources, by default None

auto_shutdownbool, optional

Whether a worker terminates itself once its command finishes, by default True. Appends shutdown -h now to the worker’s UserData.

max_idle_timeint, optional

Deprecated and ignored, by default 300. Use Parsl’s own max_idletime to reclaim idle blocks (#194).

use_public_ipsbool, optional

Whether to assign public IPs to instances, by default True

custom_amibool, optional

Whether image_id refers to a custom AMI, by default False

debugbool, optional

Whether to enable debug logging, by default False

use_spot_fleetbool, optional

Whether to use Spot Fleet for spot instances, by default False

instance_typesOptional[List[str]], optional

List of instance types to use with Spot Fleet, by default None

nodes_per_blockint, optional

Number of nodes per block, by default 1

spot_max_price_percentageOptional[int], optional

Maximum spot price as a percentage of on-demand price, by default None

warm_pool_sizeint, optional

Number of idle instances to keep for reuse, by default 0 (disabled)

warm_pool_ttlint, optional

Seconds a warm instance is kept before termination, by default 600

iam_instance_profile_arnOptional[str], optional

Instance profile ARN granting SSM access, by default None

auto_create_instance_profilebool, optional

Whether to create an instance profile with the AmazonSSMManagedInstanceCore policy when iam_instance_profile_arn is not supplied, by default False. SSM SendCommand dispatch cannot work without one of the two.

bake_amibool, optional

Whether to pre-install worker_init into a custom AMI, by default False

baked_ami_idOptional[str], optional

Pre-baked AMI to use instead of baking one, by default None

one_shotbool, optional

Whether to dispatch a single command per instance and terminate, by default False

cancel_jobs(resource_ids: List[str]) → Dict[str, str]

Cancel jobs.

Parameters

resource_idsList[str]

List of resource IDs to cancel

Returns

Dict[str, str]

Dictionary mapping resource IDs to status strings

cleanup_all() → None

Clean up all resources created by this mode.

cleanup_infrastructure() → None

Clean up infrastructure created by this mode.

This cleans up the VPC, subnet, and security group if they were created by the provider.

cleanup_resources(resource_ids: List[str]) → None

Clean up resources.

Parameters

resource_idsList[str]

List of resource IDs to clean up

get_job_status(resource_ids: List[str]) → Dict[str, str]

Get the status of jobs.

Parameters

resource_idsList[str]

List of resource IDs to check

Returns

Dict[str, str]

Dictionary mapping resource IDs to status strings

initialize() → None

Initialize standard mode infrastructure.

Raises

ResourceCreationError

If resource creation fails

property launch_template_name: str

Name of this mode’s launch template, unique per provider.

list_resources() → Dict[str, List[Dict[str, Any]]]

List all resources created by this mode.

Returns

Dict[str, List[Dict[str, Any]]]

Dictionary of resource types and their details

load_state() → bool

Load state from the state store.

Returns

bool

True if state was loaded successfully, False otherwise

save_state() → None

Save the current state to the state store.

submit_job(job_id: str, command: str, tasks_per_node: int, job_name: str | None = None) → str

Submit a job for execution.

Parameters

job_idstr

Unique identifier for the job

commandstr

Command to execute

tasks_per_nodeint

Number of tasks to run per node

job_nameOptional[str], optional

Human-readable name for the job, by default None

Returns

str

EC2 instance ID for tracking the job

Raises

OperatingModeError

If job submission fails

Detached operating mode for the EphemeralProvider.

The detached mode uses a persistent bastion host for coordinating long-running workflows, allowing the client to disconnect and reconnect to the same infrastructure.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.modes.detached.DetachedMode(provider_id: str, session: Session, state_store: StateStore, workflow_id: str | None = None, bastion_instance_type: str = 't3.micro', idle_timeout: int = 30, preserve_bastion: bool = True, bastion_host_type: str = 'cloudformation', use_spot_fleet: bool = False, instance_types: List[str] | None = None, nodes_per_block: int = 1, spot_max_price_percentage: int | None = None, bastion_id: str | None = None, bastion_instance_profile_arn: str | None = None, **kwargs: Any)

Bases: OperatingMode

Detached operating mode implementation.

In detached mode, a persistent bastion host is created to coordinate long-running workflows, allowing the client to disconnect and reconnect to the same infrastructure. The bastion host manages EC2 worker instances as needed.

Attributes

workflow_idstr

Unique identifier for the workflow

bastion_idOptional[str]

ID of the bastion host instance or CloudFormation stack

bastion_host_typestr

Type of bastion host deployment (direct or cloudformation)

bastion_instance_typestr

EC2 instance type for the bastion host

idle_timeoutint

Minutes to wait before shutting down idle resources

preserve_bastionbool

Whether to preserve the bastion host during cleanup

stack_nameOptional[str]

Name of the CloudFormation stack for the bastion host

Initialize the detached mode.

Parameters

provider_idstr

Unique identifier for the provider instance

sessionboto3.Session

AWS session for API calls

state_storeStateStore

Store for persisting state

workflow_idOptional[str], optional

Unique identifier for the workflow, by default None

bastion_instance_typestr, optional

EC2 instance type for the bastion host, by default “t3.micro”

idle_timeoutint, optional

Minutes to wait before shutting down idle resources, by default 30

preserve_bastionbool, optional

Whether to preserve the bastion host during cleanup, by default True

bastion_host_typestr, optional

Type of bastion host deployment (direct or cloudformation), by default “cloudformation”

use_spot_fleetbool, optional

Whether to use Spot Fleet for worker instances, by default False

instance_typesOptional[List[str]], optional

List of instance types to use with Spot Fleet, by default None

nodes_per_blockint, optional

Number of nodes per block, by default 1

spot_max_price_percentageOptional[int], optional

Maximum spot price as a percentage of on-demand price, by default None

bastion_instance_profile_arnOptional[str], optional

Instance profile for the bastion to carry, by default None — in which case one is created and deleted with the bastion. Supply an ARN to use your own, which this mode then never deletes. Ignored on the cloudformation path, where bastion.yml declares its own.

**kwargsAny

Additional arguments passed to the parent class

__init__(provider_id: str, session: Session, state_store: StateStore, workflow_id: str | None = None, bastion_instance_type: str = 't3.micro', idle_timeout: int = 30, preserve_bastion: bool = True, bastion_host_type: str = 'cloudformation', use_spot_fleet: bool = False, instance_types: List[str] | None = None, nodes_per_block: int = 1, spot_max_price_percentage: int | None = None, bastion_id: str | None = None, bastion_instance_profile_arn: str | None = None, **kwargs: Any) → None

Initialize the detached mode.

Parameters

provider_idstr

Unique identifier for the provider instance

sessionboto3.Session

AWS session for API calls

state_storeStateStore

Store for persisting state

workflow_idOptional[str], optional

Unique identifier for the workflow, by default None

bastion_instance_typestr, optional

EC2 instance type for the bastion host, by default “t3.micro”

idle_timeoutint, optional

Minutes to wait before shutting down idle resources, by default 30

preserve_bastionbool, optional

Whether to preserve the bastion host during cleanup, by default True

bastion_host_typestr, optional

Type of bastion host deployment (direct or cloudformation), by default “cloudformation”

use_spot_fleetbool, optional

Whether to use Spot Fleet for worker instances, by default False

instance_typesOptional[List[str]], optional

List of instance types to use with Spot Fleet, by default None

nodes_per_blockint, optional

Number of nodes per block, by default 1

spot_max_price_percentageOptional[int], optional

Maximum spot price as a percentage of on-demand price, by default None

bastion_instance_profile_arnOptional[str], optional

Instance profile for the bastion to carry, by default None — in which case one is created and deleted with the bastion. Supply an ARN to use your own, which this mode then never deletes. Ignored on the cloudformation path, where bastion.yml declares its own.

**kwargsAny

Additional arguments passed to the parent class

cancel_jobs(resource_ids: List[str]) → Dict[str, str]

Cancel jobs.

Parameters

resource_idsList[str]

List of resource IDs to cancel

Returns

Dict[str, str]

Dictionary mapping resource IDs to status strings

cleanup_all() → None

Clean up all resources created by this mode.

cleanup_infrastructure() → None

Clean up infrastructure created by this mode.

Terminates the workers, then the bastion — either by deleting its CloudFormation stack or terminating the instance, depending on bastion_host_type. The bastion is left running when preserve_bastion is set, which is what makes later reconnection possible.

The VPC, subnet, and security group are not touched: they are supplied by the caller and this mode never created them (#69). An earlier version of this docstring claimed otherwise.

cleanup_resources(resource_ids: List[str]) → None

Clean up resources.

Parameters

resource_idsList[str]

List of resource IDs to clean up

get_job_status(resource_ids: List[str]) → Dict[str, str]

Get the status of jobs.

Parameters

resource_idsList[str]

List of resource IDs to check

Returns

Dict[str, str]

Dictionary mapping resource IDs to status strings

initialize() → None

Initialize detached mode infrastructure.

Creates the bastion host for coordinating the workflow.

Raises

ResourceCreationError

If resource creation fails

list_resources() → Dict[str, List[Dict[str, Any]]]

List all resources created by this mode.

Returns

Dict[str, List[Dict[str, Any]]]

Dictionary of resource types and their details

load_state() → bool

Load state from the state store.

Returns

bool

True if state was loaded successfully, False otherwise

save_state() → None

Save the current state to the state store.

submit_job(job_id: str, command: str, tasks_per_node: int, job_name: str | None = None) → str

Submit a job for execution.

Parameters

job_idstr

Unique identifier for the job

commandstr

Command to execute

tasks_per_nodeint

Number of tasks to run per node

job_nameOptional[str], optional

Human-readable name for the job, by default None

Returns

str

Resource ID for tracking the job

Raises

OperatingModeError

If job submission fails

Serverless mode implementation for Parsl Ephemeral AWS Provider.

This mode uses AWS Lambda and ECS/Fargate for executing jobs without EC2 instances, providing cost-effective serverless execution for suitable workloads.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.modes.serverless.ServerlessMode(provider_id: str, session: Session, state_store: Any, worker_type: str = 'auto', lambda_timeout: int = 300, lambda_memory: int = 1024, lambda_runtime: str = 'python3.12', ecs_task_cpu: int = 1024, ecs_task_memory: int = 2048, ecs_container_image: str = 'python:3.12-slim', vpc_id: str | None = None, subnet_id: str | None = None, security_group_id: str | None = None, use_public_ips: bool = True, use_spot: bool = False, use_spot_fleet: bool = False, instance_types: List[str] | None = None, nodes_per_block: int = 1, spot_max_price_percentage: float | None = None, additional_tags: Dict[str, str] | None = None, debug: bool = False, compute_type: str | None = None, memory_size: int | None = None, timeout: int | None = None, lambda_code_bucket: str | None = None, **kwargs: Any)

Bases: OperatingMode

Serverless operating mode implementation.

In serverless mode, AWS Lambda and/or ECS/Fargate are used to execute tasks without any EC2 instances. This mode is suitable for event-driven or sporadic workloads with short-running tasks. It also supports EC2 SpotFleet for improved reliability and cost savings when more substantial compute resources are needed.

Attributes

worker_typestr

Type of worker to use (lambda, ecs, or auto)

lambda_timeoutint

Timeout for Lambda functions in seconds

lambda_memoryint

Memory for Lambda functions in MB

ecs_task_cpuint

CPU units for ECS tasks

ecs_task_memoryint

Memory for ECS tasks in MB

ecs_container_imagestr

Container image for ECS tasks

use_spotbool

Whether to use spot instances for ECS tasks (Fargate Spot)

use_spot_fleetbool

Whether to use Spot Fleet for EC2 instance deployment

instance_typesList[str]

List of instance types to use with Spot Fleet

nodes_per_blockint

Number of nodes per block for Spot Fleet

spot_max_price_percentageOptional[float]

Maximum spot price as percentage of on-demand price

lambda_code_bucketOptional[str]

Caller-supplied S3 bucket for staging Lambda deployment packages

lambda_managerLambdaManager

Manager for Lambda functions

ecs_managerECSManager

Manager for ECS tasks

Initialize the serverless mode.

Parameters

provider_idstr

Unique identifier for the provider instance

sessionboto3.Session

AWS session for API calls

state_storeAny

Store for persisting state

worker_typestr, optional

Type of worker to use (lambda, ecs, or auto), by default WORKER_TYPE_AUTO

lambda_timeoutint, optional

Timeout for Lambda functions in seconds, by default DEFAULT_LAMBDA_TIMEOUT

lambda_memoryint, optional

Memory for Lambda functions in MB, by default DEFAULT_LAMBDA_MEMORY

lambda_runtimestr, optional

Runtime for Lambda functions, by default DEFAULT_LAMBDA_RUNTIME

ecs_task_cpuint, optional

CPU units for ECS tasks, by default DEFAULT_ECS_CPU

ecs_task_memoryint, optional

Memory for ECS tasks in MB, by default DEFAULT_ECS_MEMORY

ecs_container_imagestr, optional

Container image for ECS tasks, by default DEFAULT_ECS_CONTAINER_IMAGE

vpc_idOptional[str], optional

Existing VPC ID to use, by default None

subnet_idOptional[str], optional

Existing subnet ID to use, by default None

security_group_idOptional[str], optional

Existing security group ID to use, by default None

use_public_ipsbool, optional

Whether to assign public IPs to ECS tasks, by default True

use_spotbool, optional

Whether to use spot instances for ECS tasks (Fargate Spot), by default False

use_spot_fleetbool, optional

Whether to use Spot Fleet for EC2 instance deployment, by default False. If True, this overrides the use_spot parameter and uses EC2 Spot Fleet instead of Fargate Spot.

instance_typesOptional[List[str]], optional

List of instance types to use with Spot Fleet, by default None. If not provided but use_spot_fleet is True, a default set of instance types will be used.

nodes_per_blockint, optional

Number of nodes per block for Spot Fleet, by default 1

spot_max_price_percentageOptional[float], optional

Maximum spot price as percentage of on-demand price, by default None. If None, AWS will use the current spot market price up to the on-demand price.

additional_tagsOptional[Dict[str, str]], optional

Tags to apply to created resources, by default None

debugbool, optional

Whether to enable debug logging, by default False

compute_typeOptional[str], optional

Compute type forwarded by EphemeralProvider (“lambda” or “ecs”). When supplied it takes precedence over worker_type; “ec2” is treated as unset since it has no meaning for serverless mode.

memory_sizeOptional[int], optional

Lambda memory in MB, forwarded by EphemeralProvider. Overrides lambda_memory when supplied.

timeoutOptional[int], optional

Lambda timeout in seconds, forwarded by EphemeralProvider. Overrides lambda_timeout when supplied.

lambda_code_bucketOptional[str], optional

Existing S3 bucket to stage Lambda deployment packages in. A caller-supplied bucket is reused as-is and never deleted; when omitted, a provider-scoped bucket is created on first use and removed by cleanup_infrastructure().

This is the surviving half of the old checkpoint_bucket parameter. Its checkpointing purpose was removed in #137 along with the unimplementable task-recovery API, but the staging override was real and is kept under a name that says what it does.

__init__(provider_id: str, session: Session, state_store: Any, worker_type: str = 'auto', lambda_timeout: int = 300, lambda_memory: int = 1024, lambda_runtime: str = 'python3.12', ecs_task_cpu: int = 1024, ecs_task_memory: int = 2048, ecs_container_image: str = 'python:3.12-slim', vpc_id: str | None = None, subnet_id: str | None = None, security_group_id: str | None = None, use_public_ips: bool = True, use_spot: bool = False, use_spot_fleet: bool = False, instance_types: List[str] | None = None, nodes_per_block: int = 1, spot_max_price_percentage: float | None = None, additional_tags: Dict[str, str] | None = None, debug: bool = False, compute_type: str | None = None, memory_size: int | None = None, timeout: int | None = None, lambda_code_bucket: str | None = None, **kwargs: Any) → None

Initialize the serverless mode.

Parameters

provider_idstr

Unique identifier for the provider instance

sessionboto3.Session

AWS session for API calls

state_storeAny

Store for persisting state

worker_typestr, optional

Type of worker to use (lambda, ecs, or auto), by default WORKER_TYPE_AUTO

lambda_timeoutint, optional

Timeout for Lambda functions in seconds, by default DEFAULT_LAMBDA_TIMEOUT

lambda_memoryint, optional

Memory for Lambda functions in MB, by default DEFAULT_LAMBDA_MEMORY

lambda_runtimestr, optional

Runtime for Lambda functions, by default DEFAULT_LAMBDA_RUNTIME

ecs_task_cpuint, optional

CPU units for ECS tasks, by default DEFAULT_ECS_CPU

ecs_task_memoryint, optional

Memory for ECS tasks in MB, by default DEFAULT_ECS_MEMORY

ecs_container_imagestr, optional

Container image for ECS tasks, by default DEFAULT_ECS_CONTAINER_IMAGE

vpc_idOptional[str], optional

Existing VPC ID to use, by default None

subnet_idOptional[str], optional

Existing subnet ID to use, by default None

security_group_idOptional[str], optional

Existing security group ID to use, by default None

use_public_ipsbool, optional

Whether to assign public IPs to ECS tasks, by default True

use_spotbool, optional

Whether to use spot instances for ECS tasks (Fargate Spot), by default False

use_spot_fleetbool, optional

Whether to use Spot Fleet for EC2 instance deployment, by default False. If True, this overrides the use_spot parameter and uses EC2 Spot Fleet instead of Fargate Spot.

instance_typesOptional[List[str]], optional

List of instance types to use with Spot Fleet, by default None. If not provided but use_spot_fleet is True, a default set of instance types will be used.

nodes_per_blockint, optional

Number of nodes per block for Spot Fleet, by default 1

spot_max_price_percentageOptional[float], optional

Maximum spot price as percentage of on-demand price, by default None. If None, AWS will use the current spot market price up to the on-demand price.

additional_tagsOptional[Dict[str, str]], optional

Tags to apply to created resources, by default None

debugbool, optional

Whether to enable debug logging, by default False

compute_typeOptional[str], optional

Compute type forwarded by EphemeralProvider (“lambda” or “ecs”). When supplied it takes precedence over worker_type; “ec2” is treated as unset since it has no meaning for serverless mode.

memory_sizeOptional[int], optional

Lambda memory in MB, forwarded by EphemeralProvider. Overrides lambda_memory when supplied.

timeoutOptional[int], optional

Lambda timeout in seconds, forwarded by EphemeralProvider. Overrides lambda_timeout when supplied.

lambda_code_bucketOptional[str], optional

Existing S3 bucket to stage Lambda deployment packages in. A caller-supplied bucket is reused as-is and never deleted; when omitted, a provider-scoped bucket is created on first use and removed by cleanup_infrastructure().

This is the surviving half of the old checkpoint_bucket parameter. Its checkpointing purpose was removed in #137 along with the unimplementable task-recovery API, but the staging override was real and is kept under a name that says what it does.

cancel_jobs(resource_ids: List[str]) → Dict[str, str]

Cancel jobs.

Parameters

resource_idsList[str]

List of resource IDs to cancel

Returns

Dict[str, str]

Dictionary mapping resource IDs to status strings

cleanup_all() → None

Clean up all resources created by this mode.

cleanup_infrastructure() → None

Clean up infrastructure created by this mode.

The VPC, subnet, and security group are supplied by the caller and are never created — or deleted — by this mode. Only the Lambda functions, ECS tasks, and Spot Fleet resources created here are removed.

cleanup_resources(resource_ids: List[str]) → None

Clean up resources.

Parameters

resource_idsList[str]

List of resource IDs to clean up

get_job_status(resource_ids: List[str]) → Dict[str, str]

Get the status of jobs.

Parameters

resource_idsList[str]

List of resource IDs to check

Returns

Dict[str, str]

Dictionary mapping resource IDs to status strings

initialize() → None

Initialize serverless mode.

Verifies the caller-supplied network resources (when the worker type needs them) and initializes the Lambda and/or ECS managers. Network resources are never created by this mode.

Raises

ResourceNotFoundError

If a configured VPC, subnet, or security group is missing.

ResourceCreationError

If initialization fails

list_resources() → Dict[str, List[Dict[str, Any]]]

List all resources created by this mode.

Returns

Dict[str, List[Dict[str, Any]]]

Dictionary of resource types and their details

load_state() → bool

Load state from the state store.

Returns

bool

True if state was loaded successfully, False otherwise

save_state() → None

Save the current state to the state store.

submit_job(job_id: str, command: str, tasks_per_node: int, job_name: str | None = None) → str

Submit a job for execution.

Parameters

job_idstr

Unique identifier for the job

commandstr

Command to execute

tasks_per_nodeint

Number of tasks to run per node

job_nameOptional[str], optional

Human-readable name for the job, by default None

Returns

str

Resource ID for tracking the job

Raises

OperatingModeError

If job submission fails

Compute

Fleet-based compute resource implementation for Parsl Ephemeral AWS Provider.

Provides multi-pool spot instance management, which is more reliable than individual spot requests: a fleet draws from several instance types at once, so a single exhausted capacity pool does not fail the request.

Built on EC2 Fleet (CreateFleet) since #86. It previously used Spot Fleet (RequestSpotFleet), which AWS describes as “a legacy API with no planned investment”, recommending EC2 Fleet or EC2 Auto Scaling instead. The class name is unchanged to keep the use_spot_fleet provider kwarg and the persisted state documents working across the upgrade.

Fleet type instant is used throughout: it returns the launched instance IDs synchronously, so a block knows its instances without polling. See parsl_ephemeral_provider.utils.aws.create_ec2_fleet() for the parameters this fleet type rejects, and why capacity rebalancing is not among the options.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.compute.spot_fleet.SpotFleetManager(provider: Any)

Bases: object

Manager for AWS EC2 Fleet compute resources.

Requests instances from several instance types at once, so an exhausted capacity pool degrades the fleet rather than failing it, and the allocation strategy can pick the pools least likely to be interrupted.

Named for the legacy Spot Fleet API it was originally built on. It now calls CreateFleet (#86); the name is retained because it is reachable through the public use_spot_fleet provider kwarg and appears in persisted state.

Initialize the fleet manager.

Parameters

providerEphemeralProvider

The provider instance

__init__(provider: Any) → None

Initialize the fleet manager.

Parameters

providerEphemeralProvider

The provider instance

cleanup_all_resources() → None

Clean up all AWS resources created by this manager.

create_blocks(count: int) → Dict[str, Dict[str, Any]]

Create compute blocks, one EC2 Fleet each.

Parameters

countint

Number of blocks to create

Returns

Dict[str, Dict[str, Any]]

Dictionary mapping block IDs to block information

get_block_status(block_id: str) → str

Get the status of a block.

An instant fleet does not maintain capacity, so its FleetState stays active for the life of the fleet regardless of what happened to the instances. The block’s status therefore comes from the instances, with the fleet state consulted only for the terminal cases.

Parameters

block_idstr

ID of the block to check

Returns

str

Block status

get_instance_private_ip(instance_id: str) → str | None

Get the private IP address of an instance.

Parameters

instance_idstr

ID of the instance

Returns

Optional[str]

Private IP address, or None if not available

get_instance_public_ip(instance_id: str) → str | None

Get the public IP address of an instance.

Parameters

instance_idstr

ID of the instance

Returns

Optional[str]

Public IP address, or None if not available

terminate_block(block_id: str) → None

Terminate a compute block.

Parameters

block_idstr

ID of the block to terminate

Raises

SpotFleetThrottlingError

If AWS throttled the deletion. Carries retry_after, so a caller can wait the interval AWS named rather than retry blind.

SpotFleetError

If EC2 refused the deletion for any other reason.

ResourceCleanupError

For a failure that is not EC2 refusing the deletion.

Detection of AWS spot instance interruptions.

This module detects that AWS is reclaiming a spot instance and calls the handlers registered for it. It does not decide what to do about it: the response lives on OperatingMode, which marks the doomed block interrupted so Parsl re-runs its tasks.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.compute.spot_interruption.SpotInterruptionMonitor(session: Session, check_interval: int = 30, lead_time: int = 120, provider_id: str | None = None, use_event_bridge: bool = True)

Bases: object

Monitor for AWS spot instance interruption notices.

The SpotInterruptionMonitor checks for spot instance interruption notices and executes recovery actions when interruptions are detected. It can monitor both individual spot instances and spot fleet requests.

Detection runs on two tracks. The EventBridge warning is the one that matters: it arrives roughly two minutes before the reclaim, with the instance still running, so the block can be marked STATUS_INTERRUPTED while the executor still has time to stop dispatching into it. The EC2-state poll is kept as a fallback for when the notifier could not be created (missing events/sqs permissions, say), but it can only ever report an interruption post-facto, once the instance has already reached shutting-down – by which time work has been dispatched to a worker that is already gone.

Attributes

sessionboto3.Session

AWS session for making API calls

check_intervalint

Interval in seconds between checks for interruption notices

lead_timeint

Minimum time in seconds we want for recovery before instance termination

instance_handlersDict[str, Callable]

Mapping of instance IDs to handler functions

fleet_handlersDict[str, Callable]

Mapping of fleet request IDs to handler functions

monitoring_threadOptional[threading.Thread]

Thread for background monitoring

stop_eventthreading.Event

Event to signal thread termination

warning_rule_nameOptional[str]

EventBridge rule delivering interruption warnings, once created.

warning_queue_urlOptional[str]

SQS queue the rule delivers to, once created.

Initialize the SpotInterruptionMonitor.

Parameters

sessionboto3.Session

AWS session for making API calls

check_intervalint, optional

Interval in seconds between checks for interruption notices

lead_timeint, optional

Minimum time in seconds we want for recovery before instance termination

provider_idOptional[str], optional

Names the EventBridge rule and SQS queue, keeping them unique per provider. Falls back to a random suffix when omitted.

use_event_bridgebool, optional

Whether to create the EventBridge notifier that supplies the two-minute advance warning. Set False to rely solely on the post-facto EC2-state poll – useful when the caller’s IAM policy grants no events/sqs access.

__init__(session: Session, check_interval: int = 30, lead_time: int = 120, provider_id: str | None = None, use_event_bridge: bool = True) → None

Initialize the SpotInterruptionMonitor.

Parameters

sessionboto3.Session

AWS session for making API calls

check_intervalint, optional

Interval in seconds between checks for interruption notices

lead_timeint, optional

Minimum time in seconds we want for recovery before instance termination

provider_idOptional[str], optional

Names the EventBridge rule and SQS queue, keeping them unique per provider. Falls back to a random suffix when omitted.

use_event_bridgebool, optional

Whether to create the EventBridge notifier that supplies the two-minute advance warning. Set False to rely solely on the post-facto EC2-state poll – useful when the caller’s IAM policy grants no events/sqs access.

deregister_fleet(fleet_request_id: str) → None

Stop monitoring a spot fleet.

Parameters

fleet_request_idstr

ID of the spot fleet request to stop monitoring

deregister_instance(instance_id: str) → None

Stop monitoring a spot instance.

Parameters

instance_idstr

ID of the spot instance to stop monitoring

register_fleet(fleet_request_id: str, handler: Callable[[str, List[str], Dict[str, Any]], None]) → None

Register a spot fleet to be monitored.

Parameters

fleet_request_idstr

ID of the spot fleet request to monitor

handlerCallable[[str, List[str], Dict[str, Any]], None]

Function to call when interruption is detected, receives fleet_request_id, list of affected instance_ids, and event details

register_instance(instance_id: str, handler: Callable[[str, Dict[str, Any]], None]) → None

Register a spot instance to be monitored.

Parameters

instance_idstr

ID of the spot instance to monitor

handlerCallable[[str, Dict[str, Any]], None]

Function to call when interruption is detected, receives instance_id and event details

start_monitoring() → None

Start background monitoring for spot interruption notices.

Creates the EventBridge notifier first, when enabled. A failure there is logged and not raised: losing the advance warning degrades this to the post-facto EC2-state poll, which is worse but still functional, and is not a reason to fail the workflow that was about to run.

stop_monitoring() → None

Stop background monitoring, and delete the EventBridge notifier.

The notifier is torn down even when no thread was running, so a monitor that was stopped twice – or whose thread died – still cleans up the rule and queue it created rather than leaking them.

Lambda function compute implementation for Parsl Ephemeral AWS Provider.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.compute.lambda_func.LambdaManager(provider: Any)

Bases: object

Manager for AWS Lambda compute resources.

Initialize the Lambda manager.

Parameters

providerEphemeralProvider

The provider instance

__init__(provider: Any) → None

Initialize the Lambda manager.

Parameters

providerEphemeralProvider

The provider instance

cleanup_all_resources() → None

Clean up all AWS resources created by this manager.

get_job_status(function_name: str, request_id: str) → str

Get the status of a job.

Parameters

function_namestr

Name of the Lambda function

request_idstr

Request ID from the function invocation

Returns

str

Job status

submit_job(job_id: str, command: str) → Dict[str, Any]

Submit a job for execution.

Parameters

job_idstr

ID of the job

commandstr

Command to execute

Returns

Dict[str, Any]

Dictionary containing job information

ECS/Fargate compute implementation for Parsl Ephemeral AWS Provider.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.compute.ecs.ECSManager(provider: Any)

Bases: object

Manager for AWS ECS/Fargate compute resources.

Initialize the ECS manager.

Parameters

providerEphemeralProvider

The provider instance

__init__(provider: Any) → None

Initialize the ECS manager.

Parameters

providerEphemeralProvider

The provider instance

cancel_job(cluster: str, task_id: str) → None

Cancel a job.

Parameters

clusterstr

Name of the ECS cluster

task_idstr

ID of the ECS task

cleanup_all_resources() → None

Clean up all AWS resources created by this manager.

get_job_status(cluster: str, task_id: str) → str

Get the status of a job.

Parameters

clusterstr

Name of the ECS cluster

task_idstr

ID of the ECS task

Returns

str

Job status

submit_job(job_id: str, command: str, tasks_per_node: int) → Dict[str, Any]

Submit a job for execution.

Parameters

job_idstr

ID of the job

commandstr

Command to execute

tasks_per_nodeint

Number of tasks per node

Returns

Dict[str, Any]

Dictionary containing job information

State persistence

Base state store interface for the EphemeralProvider.

State documents are addressed by a state key. The provider and its operating mode persist different, partially overlapping sets of fields, so each owns its own key: without that separation the two full-document writes overwrite each other and mode-only fields (the baked AMI ID, the warm-pool list) or provider-only fields (job_map) are silently destroyed (#78).

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

parsl_ephemeral_provider.state.base.STATE_KEY_MODE = 'mode'

State key owned by the OperatingMode (network IDs, baked AMI, warm pool, …)

parsl_ephemeral_provider.state.base.STATE_KEY_PROVIDER = 'provider'

State key owned by EphemeralProvider (resources, job_map, …)

class parsl_ephemeral_provider.state.base.StateStore(provider_id: str)

Bases: ABC

Abstract base class for provider state stores.

A state store persists and retrieves state documents, each addressed by a state_key. Different implementations store them in different places — local files, AWS Parameter Store, or S3.

Attributes

provider_idstr

Unique identifier for the provider instance

Initialize the state store.

Parameters

provider_idstr

Unique identifier for the provider instance

__init__(provider_id: str) → None

Initialize the state store.

Parameters

provider_idstr

Unique identifier for the provider instance

abstractmethod delete_state(state_key: str) → None

Delete a state document.

Deleting a key that does not exist is not an error.

Parameters

state_keystr

Key to delete the document for

Raises

StateStoreError

If deleting state fails

abstractmethod load_state(state_key: str) → Dict[str, Any] | None

Load a state document.

Parameters

state_keystr

Key to load the document from

Returns

Optional[Dict[str, Any]]

State document if it exists, None otherwise

Raises

StateStoreError

If loading state fails

abstractmethod save_state(state_key: str, state_data: Dict[str, Any]) → None

Save a state document.

Writing one key must leave the other keys in the store untouched.

Parameters

state_keystr

Key to store the document under

state_dataDict[str, Any]

State document to save

Raises

StateStoreError

If saving state fails

parsl_ephemeral_provider.state.base.get_provider_id(provider: Any) → str

Return a provider’s own identifier.

parsl_ephemeral_provider.state.base.get_workflow_id(provider: Any) → str

Return a provider’s workflow identifier, for tagging stored state.

workflow_id is the convention the compute managers use; the provider itself only guarantees provider_id. Falls back to "unknown" so a missing identifier degrades a tag rather than failing the save.

parsl_ephemeral_provider.state.base.resolve_session(provider: Any) → Session

Return a boto3 session for talking to a provider’s account.

Prefers the session the provider already built — it resolved credentials once, correctly, via utils.aws.create_session(). Only when there is none does this fall back to assembling a session from whatever credential attributes the object happens to carry.

Every attribute is read with getattr because this is called with either an EphemeralProvider or an OperatingMode, and neither defines the full set. Reading them directly is what made the AWS state stores raise AttributeError on construction (#77).

Parameters

providerAny

An EphemeralProvider or OperatingMode

Returns

boto3.Session

A session bound to the provider’s region

File-based state store for the EphemeralProvider.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.state.file.FileStateStore(file_path: str, provider_id: str)

Bases: StateStore

File-based state store implementation.

Stores state in a single local JSON file. Each state_key is a top-level sub-document under _states, so writing one key preserves the others:

{"_version": 2, "_states": {"provider": {...}, "mode": {...}}}

Files written before v0.7.0 hold a single flat document with no _states wrapper. Those are read back under every key — the provider and the mode each take the fields they recognise — and the first write upgrades the file to the keyed layout, seeding both keys from the flat document so the writer’s counterpart can still find its fields afterwards.

Attributes

file_pathstr

Path to the state file

provider_idstr

Unique identifier for the provider instance

Initialize the file state store.

Parameters

file_pathstr

Path to the state file

provider_idstr

Unique identifier for the provider instance

__init__(file_path: str, provider_id: str) → None

Initialize the file state store.

Parameters

file_pathstr

Path to the state file

provider_idstr

Unique identifier for the provider instance

delete_state(state_key: str) → None

Delete the state document stored under state_key.

The file itself is removed once the last key is gone.

Parameters

state_keystr

Key to delete the document for

Raises

StateStoreError

If deleting state fails

load_state(state_key: str) → Dict[str, Any] | None

Load the state document stored under state_key.

Parameters

state_keystr

Key to load the document from

Returns

Optional[Dict[str, Any]]

State document if present, None otherwise. A flat pre-v0.7.0 file is returned whole for any key.

Raises

StateDeserializationError

If deserializing state fails

StateStoreError

If loading state fails

save_state(state_key: str, state_data: Dict[str, Any]) → None

Save a state document under state_key.

Read-modify-write: the other keys in the file are preserved.

Parameters

state_keystr

Key to store the document under

state_dataDict[str, Any]

State document to save

Raises

StateSerializationError

If serializing state fails

StateStoreError

If saving state fails

S3 state implementation for Parsl Ephemeral AWS Provider.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.state.s3.S3State(provider: Any, bucket_name: str, key_prefix: str = 'parsl/workflows', create_bucket_if_not_exists: bool = False)

Bases: StateStore

AWS S3 implementation of state persistence.

Initialize S3 state.

Parameters

providerEphemeralProvider

The provider instance

bucket_namestr

Name of the S3 bucket to use

key_prefixstr, optional

Prefix for S3 keys, by default ‘parsl/workflows’

create_bucket_if_not_existsbool, optional

Whether to create the bucket if it doesn’t exist, by default False

__init__(provider: Any, bucket_name: str, key_prefix: str = 'parsl/workflows', create_bucket_if_not_exists: bool = False) → None

Initialize S3 state.

Parameters

providerEphemeralProvider

The provider instance

bucket_namestr

Name of the S3 bucket to use

key_prefixstr, optional

Prefix for S3 keys, by default ‘parsl/workflows’

create_bucket_if_not_existsbool, optional

Whether to create the bucket if it doesn’t exist, by default False

cleanup_workflow_states() → None

Clean up all states for the current workflow.

delete_bucket_if_empty() → bool

Delete the S3 bucket if it’s empty.

Returns

bool

Whether the bucket was deleted

delete_state(state_key: str) → None

Delete provider state from S3.

Parameters

state_keystr

Key to delete the state for

list_states(prefix: str) → Dict[str, Dict[str, Any]]

List all states with a given prefix.

Parameters

prefixstr

Prefix to list states for

Returns

Dict[str, Dict[str, Any]]

Dictionary mapping state keys to state data

load_state(state_key: str) → Dict[str, Any] | None

Load provider state from S3.

Parameters

state_keystr

Key to load the state from

Returns

Optional[Dict[str, Any]]

Loaded state data, or None if not found

save_state(state_key: str, state_data: Dict[str, Any]) → None

Save provider state in S3.

Parameters

state_keystr

Key to store the state under

state_dataDict[str, Any]

State data to save

parsl_ephemeral_provider.state.s3.S3StateStore

alias of S3State

Parameter Store state implementation for Parsl Ephemeral AWS Provider.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.state.parameter_store.ParameterStoreState(provider: Any, prefix: str = '/parsl/workflows', use_secure_string: bool = False)

Bases: StateStore

AWS Parameter Store implementation of state persistence.

Initialize Parameter Store state.

Parameters

providerEphemeralProvider

The provider instance

prefixstr, optional

Prefix for parameter names, by default ‘/parsl/workflows’

use_secure_stringbool, optional

Whether to use SecureString parameter type, by default False

__init__(provider: Any, prefix: str = '/parsl/workflows', use_secure_string: bool = False) → None

Initialize Parameter Store state.

Parameters

providerEphemeralProvider

The provider instance

prefixstr, optional

Prefix for parameter names, by default ‘/parsl/workflows’

use_secure_stringbool, optional

Whether to use SecureString parameter type, by default False

cleanup_workflow_states() → None

Clean up all states for the current workflow.

delete_state(state_key: str) → None

Delete provider state from Parameter Store.

Parameters

state_keystr

Key to delete the state for

list_states(prefix: str) → Dict[str, Dict[str, Any]]

List all states with a given prefix.

Parameters

prefixstr

Prefix to list states for

Returns

Dict[str, Dict[str, Any]]

Dictionary mapping state keys to state data

load_state(state_key: str) → Dict[str, Any] | None

Load provider state from Parameter Store.

Parameters

state_keystr

Key to load the state from

Returns

Optional[Dict[str, Any]]

Loaded state data, or None if not found

save_state(state_key: str, state_data: Dict[str, Any]) → None

Save provider state in Parameter Store.

Parameters

state_keystr

Key to store the state under

state_dataDict[str, Any]

State data to save

parsl_ephemeral_provider.state.parameter_store.ParameterStoreStateStore

alias of ParameterStoreState

Support modules

Clean constants for the EphemeralProvider.

No legacy garbage, just what’s actually needed.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

Custom exceptions for the EphemeralProvider.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

exception parsl_ephemeral_provider.exceptions.AMINotFoundError

Bases: ResourceCreationError

Specified AMI not found.

exception parsl_ephemeral_provider.exceptions.AWSAuthenticationError

Bases: AWSConnectionError

AWS authentication failure.

exception parsl_ephemeral_provider.exceptions.AWSConnectionError

Bases: ProviderError

Error connecting to AWS services.

exception parsl_ephemeral_provider.exceptions.BastionHostError

Bases: ResourceCreationError

Error managing bastion hosts.

exception parsl_ephemeral_provider.exceptions.CleanupError

Bases: ProviderError

Error cleaning up resources.

exception parsl_ephemeral_provider.exceptions.CloudFormationError

Bases: ResourceCreationError

Error in CloudFormation stack operations.

exception parsl_ephemeral_provider.exceptions.ConfigurationError

Bases: ProviderConfigurationError

Error in provider configuration.

exception parsl_ephemeral_provider.exceptions.CredentialResolutionError(message: str)

Bases: NoCredentialsError

No usable AWS credentials could be resolved, with a reason.

botocore’s NoCredentialsError derives from BotoCoreError, whose __init__ accepts keyword arguments only and formats them into a fixed class-level fmt — so NoCredentialsError("why") raises TypeError: BotoCoreError.__init__() takes 1 positional argument but 2 were given and the reason never reaches the caller. Every raise site in security/credential_manager.py did exactly that, turning each of the seven distinct credential failures into the same opaque TypeError.

Subclassing keeps except NoCredentialsError handlers working — compute/{ec2,ecs,lambda_func,spot_fleet}.py and error_handling.py:269 all catch the botocore class and must continue to — while allowing the message through.

fmt = '{message}'

exception parsl_ephemeral_provider.exceptions.EC2InstanceError

Bases: ResourceCreationError

Error managing EC2 instances.

exception parsl_ephemeral_provider.exceptions.ECSTaskError

Bases: ResourceCreationError

Error managing ECS tasks.

exception parsl_ephemeral_provider.exceptions.EphemeralAWSError

Bases: Exception

Base class for all EphemeralProvider exceptions.

exception parsl_ephemeral_provider.exceptions.InvalidStateError

Bases: ProviderError

Provider is in an invalid state for the requested operation.

exception parsl_ephemeral_provider.exceptions.JobCancellationError

Bases: ProviderError

Error cancelling a job.

exception parsl_ephemeral_provider.exceptions.JobExecutionError

Bases: ProviderError

Error executing a job.

exception parsl_ephemeral_provider.exceptions.JobSubmissionError

Bases: ProviderError

Error submitting job for execution.

exception parsl_ephemeral_provider.exceptions.LambdaFunctionError

Bases: ResourceCreationError

Error managing Lambda functions.

exception parsl_ephemeral_provider.exceptions.NetworkCreationError

Bases: ResourceCreationError

Error creating network resources.

exception parsl_ephemeral_provider.exceptions.OperatingModeError

Bases: ProviderError

Error in operating mode functionality.

exception parsl_ephemeral_provider.exceptions.ProviderConfigurationError

Bases: ProviderError

Error in provider configuration.

exception parsl_ephemeral_provider.exceptions.ProviderError

Bases: EphemeralAWSError

General provider error.

exception parsl_ephemeral_provider.exceptions.ResourceCleanupError

Bases: ResourceDeletionError

Error cleaning up AWS resources.

exception parsl_ephemeral_provider.exceptions.ResourceCreationError

Bases: ProviderError

Error creating AWS resources.

exception parsl_ephemeral_provider.exceptions.ResourceDeletionError

Bases: ProviderError

Error deleting AWS resources.

exception parsl_ephemeral_provider.exceptions.ResourceNotFoundError

Bases: ProviderError

Requested AWS resource not found.

exception parsl_ephemeral_provider.exceptions.SecurityGroupError

Bases: ResourceCreationError

Error managing security groups.

exception parsl_ephemeral_provider.exceptions.SpotFleetError

Bases: EC2InstanceError

Error managing EC2 Spot Fleet.

exception parsl_ephemeral_provider.exceptions.SpotFleetRequestError

Bases: SpotFleetError

Error creating or managing Spot Fleet requests.

exception parsl_ephemeral_provider.exceptions.SpotFleetThrottlingError(message='AWS Spot Fleet API request was throttled', operation=None, retry_after=None)

Bases: SpotFleetError

AWS API throttling for Spot Fleet operations.

exception parsl_ephemeral_provider.exceptions.SpotInstanceError

Bases: EC2InstanceError

Error managing spot instances.

exception parsl_ephemeral_provider.exceptions.SpotInterruptionError

Bases: SpotInstanceError

Error related to spot instance interruption.

exception parsl_ephemeral_provider.exceptions.StateDeserializationError

Bases: StateStoreError

Error deserializing state data.

exception parsl_ephemeral_provider.exceptions.StateError

Bases: StateStoreError

General state management error - alias for compatibility.

exception parsl_ephemeral_provider.exceptions.StateSerializationError

Bases: StateStoreError

Error serializing state data.

exception parsl_ephemeral_provider.exceptions.StateStoreError

Bases: ProviderError

Error in state store operations.

exception parsl_ephemeral_provider.exceptions.TaggingError

Bases: ProviderError

Error tagging AWS resources.

exception parsl_ephemeral_provider.exceptions.TaskTimeoutError

Bases: JobExecutionError

Task execution timeout.

Enhanced error handling and recovery framework for Parsl Ephemeral AWS Provider.

This module provides robust error handling, retry mechanisms, and recovery strategies for AWS operations and provider state management.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

class parsl_ephemeral_provider.error_handling.ErrorAnalyzer

Bases: object

Analyzes errors and determines appropriate recovery actions.

Initialize error analyzer.

__init__()

Initialize error analyzer.

analyze_error(exception: Exception, context: ErrorContext) → tuple[ErrorSeverity, RecoveryAction]

Analyze an error and determine appropriate response.

Parameters

exceptionException

Exception to analyze

contextErrorContext

Context of the error

Returns

tuple[ErrorSeverity, RecoveryAction]

Error severity and recommended recovery action

should_escalate(error_record: ErrorRecord, similar_errors: int) → bool

Determine if error should be escalated.

Parameters

error_recordErrorRecord

Current error record

similar_errorsint

Number of similar errors recently

Returns

bool

True if error should be escalated

class parsl_ephemeral_provider.error_handling.ErrorContext(operation: str, resource_type: str, resource_id: str | None = None, region: str | None = None, attempt: int = 1, start_time: float = <factory>, metadata: Dict[str, ~typing.Any]=<factory>)

Bases: object

Context information for error handling.

attempt: int = 1

elapsed_time() → float

Get elapsed time since operation start.

Returns

float

Elapsed time in seconds

metadata: Dict[str, Any]

operation: str

region: str | None = None

resource_id: str | None = None

resource_type: str

start_time: float

class parsl_ephemeral_provider.error_handling.ErrorRecord(exception: Exception, context: ErrorContext, severity: ErrorSeverity, recovery_action: RecoveryAction, timestamp: float = <factory>, resolved: bool = False, resolution_time: float | None = None)

Bases: object

Record of an error for analysis and reporting.

context: ErrorContext

exception: Exception

mark_resolved() → None

Mark error as resolved.

recovery_action: RecoveryAction

resolution_duration() → float | None

Get time taken to resolve error.

Returns

Optional[float]

Resolution duration in seconds, None if not resolved

resolution_time: float | None = None

resolved: bool = False

severity: ErrorSeverity

timestamp: float

class parsl_ephemeral_provider.error_handling.ErrorRecoveryHandler

Bases: object

Handles error recovery and fallback strategies.

Initialize recovery handler.

__init__()

Initialize recovery handler.

attempt_recovery(error_record: ErrorRecord, fallback_params: Dict[str, Any] = None) → bool

Attempt to recover from an error.

Parameters

error_recordErrorRecord

Error to recover from

fallback_paramsDict[str, Any], optional

Parameters for fallback strategies

Returns

bool

True if recovery was successful

class parsl_ephemeral_provider.error_handling.ErrorSeverity(*values)

Bases: Enum

Error severity levels.

CRITICAL = 'critical'

HIGH = 'high'

LOW = 'low'

MEDIUM = 'medium'

class parsl_ephemeral_provider.error_handling.RecoveryAction(*values)

Bases: Enum

Recovery actions for error handling.

ABORT = 'abort'

CLEANUP = 'cleanup'

FALLBACK = 'fallback'

IGNORE = 'ignore'

RETRY = 'retry'

class parsl_ephemeral_provider.error_handling.RetryConfig(max_attempts: int = 3, base_delay: float = 1.0, max_delay: float = 60.0, exponential_backoff: bool = True, jitter: bool = True, jitter_factor: float = 0.1, retry_on_exceptions: List[Type[Exception]] = <factory>, retry_on_status_codes: List[int] = <factory>)

Bases: object

Configuration for retry behavior.

base_delay: float = 1.0

exponential_backoff: bool = True

get_delay(attempt: int) → float

Calculate delay for retry attempt.

Parameters

attemptint

Current attempt number (1-based)

Returns

float

Delay in seconds

jitter: bool = True

jitter_factor: float = 0.1

max_attempts: int = 3

max_delay: float = 60.0

retry_on_exceptions: List[Type[Exception]]

retry_on_status_codes: List[int]

should_retry(exception: Exception, attempt: int) → bool

Determine if an exception should trigger a retry.

Parameters

exceptionException

Exception to evaluate

attemptint

Current attempt number

Returns

bool

True if should retry

class parsl_ephemeral_provider.error_handling.RobustErrorHandler(retry_config: RetryConfig | None = None)

Bases: object

Main error handling coordinator.

Initialize robust error handler.

Parameters

retry_configOptional[RetryConfig]

Retry configuration, uses default if None

__init__(retry_config: RetryConfig | None = None)

Initialize robust error handler.

Parameters

retry_configOptional[RetryConfig]

Retry configuration, uses default if None

error_history: List[ErrorRecord]

get_error_statistics(time_window: float = 3600) → Dict[str, Any]

Get error statistics for a time window.

Parameters

time_windowfloat

Time window in seconds (default: 1 hour)

Returns

Dict[str, Any]

Error statistics

handle_error(exception: Exception, context: ErrorContext, fallback_params: Dict[str, Any] | None = None) → ErrorRecord

Handle an error with analysis and recovery.

Parameters

exceptionException

Exception that occurred

contextErrorContext

Context of the operation

fallback_paramsOptional[Dict[str, Any]]

Parameters for fallback strategies

Returns

ErrorRecord

Record of the error and handling

parsl_ephemeral_provider.error_handling.poll_until(predicate: Callable[[], T | None], *, timeout: float, description: str, retry_config: RetryConfig | None = None, on_error: Callable[[Exception], None] | None = None) → T

Poll predicate until it returns a truthy value, with backoff and jitter.

This is the framework’s entry point for a success-poll, which is a different shape from retry_with_backoff() and cannot be expressed with it (#91). The decorator retries a call that raised; here the call succeeds and returns a not-yet answer – an instance that has not appeared in describe_instance_information, a fleet block that has not reached running. Nothing is thrown, so the decorator would never fire and the loop would run exactly once.

Before this existed, modes/ hand-rolled three of these with flat time.sleep(10)/sleep(15) intervals, which is the concrete debt #91 tracked: no jitter, so N providers started together poll AWS in lockstep, and no shared notion of a bounded wait.

Parameters

predicateCallable[[], Optional[T]]

Called once per attempt. Return a truthy value to stop and have it returned; return None or any falsey value to keep waiting. Raising is treated as “not yet” – see on_error.

timeoutfloat

Total seconds to keep polling before giving up. This bounds wall-clock time, not attempt count: unlike RetryConfig.max_attempts, a caller waiting for a 10-minute boot wants a deadline rather than a number of tries.

descriptionstr

What is being waited for, used in the timeout message and debug logs. Phrase it as a noun so the message reads “timed out waiting for {…}”.

retry_configOptional[RetryConfig]

Supplies the delay schedule via RetryConfig.get_delay(), so a poll gets the same exponential backoff and jitter as a retry. Defaults to RetryConfig(). max_attempts is deliberately not consulted – timeout is the bound here.

on_errorOptional[Callable[[Exception], None]]

Called with any exception the predicate raises, then polling continues. Use it to log at the level the caller wants. When omitted, exceptions are logged at debug: a poll’s early attempts are expected to fail (the resource does not exist yet), so warning on each one turns normal operation into a wall of noise.

Returns

T

The first truthy value predicate returned.

Raises

TimeoutError

If timeout elapses with no truthy result. Callers that owe their own exception type should catch this and re-raise; modes/standard.py converts it to OperatingModeError.

parsl_ephemeral_provider.error_handling.retry_with_backoff(retry_config: RetryConfig | None = None, error_handler: RobustErrorHandler | None = None)

Decorator for adding retry behavior with exponential backoff.

Parameters

retry_configOptional[RetryConfig]

Retry configuration

error_handlerOptional[RobustErrorHandler]

Error handler for comprehensive error management

Returns

Callable

Decorated function with retry behavior

AWS utility functions for the EphemeralProvider.

SPDX-License-Identifier: Apache-2.0 SPDX-FileCopyrightText: 2025-2026 Scott Friedman and Project Contributors

parsl_ephemeral_provider.utils.aws.BASTION_INLINE_POLICY_NAME = 'BastionHostPolicy'

Name of the bastion role’s inline policy. Matches bastion.yml’s PolicyName, so the two paths present the same thing to an operator reading the console.

parsl_ephemeral_provider.utils.aws.EC2_ASSUME_ROLE_POLICY: Dict[str, Any] = {'Statement': [{'Action': 'sts:AssumeRole', 'Effect': 'Allow', 'Principal': {'Service': 'ec2.amazonaws.com'}}], 'Version': '2012-10-17'}

Trust policy letting EC2 assume a role, so an instance can carry it.

Shared by the worker (SSM) role and the bastion role rather than written out at each site: the two roles differ in what they may do, never in who may assume them, and a copied-out trust policy is where that distinction quietly erodes.

parsl_ephemeral_provider.utils.aws.architecture_for_instance_type(instance_type: str) → str

Return the CPU architecture an instance type needs an AMI for.

An AMI is architecture-specific, so launching a Graviton instance with an x86_64 image fails. Nothing in this package distinguished the two before #84, which made every arm64 instance type unusable.

The family suffix is the signal: AWS appends g to the generation of every Graviton family (c7g, m8g, r7gd, c8gn, …) and to no x86_64 family. Validated against describe_instance_types for all 1,346 types AWS offers in us-east-1: 396 arm64 and 950 x86_64, zero mistakes.

The only exceptions are the eight mac*.metal types, which report arm64_mac and need a macOS AMI rather than AL2023 – so classifying them as x86_64 is no worse than the arm64 answer would be. A caller wanting a Mac instance must pass image_id explicitly either way.

Parameters

instance_typestr

EC2 instance type, e.g. "c7g.xlarge" or "t3.micro".

Returns

str

"arm64" or "x86_64".

parsl_ephemeral_provider.utils.aws.bastion_instance_profile_names(name_suffix: str) → Tuple[str, str]

Return the bastion (role_name, profile_name) pair for name_suffix.

Derived rather than stored, for the same reason as ssm_instance_profile_names. Kept separate from that pair because the two roles carry different permissions: the worker role only needs SSM, while the bastion role can launch and terminate instances.

IAM caps both names at 64 characters. name_suffix is a provider ID — a UUID, 36 characters — so the longest name here is 58.

parsl_ephemeral_provider.utils.aws.bastion_role_policy(region: str, account_id: str, workflow_id: str) → Dict[str, Any]

Return the inline policy granting a bastion exactly what it calls.

bastion.yml’s BastionHostPolicy is the same set, and the two are meant to stay in step: a bastion deployed by CloudFormation and one launched by RunInstances run the identical manager script, so a permission missing from either is a bug in that path only, which is the hardest kind to notice.

The list is derived from what _get_bastion_manager_script actually calls, not from what a bastion plausibly needs, and a test asserts the correspondence in both directions. Enumerating the script found the CloudFormation policy both too narrow and too wide: it omitted ec2:RunInstances, the two launch-template calls and all three fleet calls — so a spot-fleet job on the CFN path could not launch a worker either — while granting ec2:StartInstances, ec2:StopInstances, ec2:DescribeInstanceStatus and ec2:DescribeTags, none of which the script calls (the bastion terminates workers rather than stopping them, and it reads worker state through DescribeInstances alone).

Three actions need explaining:

• iam:PassRole is absent, deliberately. The manager launches workers with no instance profile, so it passes no role. Adding it would let a compromised bastion attach any passable role to an instance it launches, which is a privilege-escalation primitive rather than a convenience.

• ec2:CreateTags is unavoidably broad. The script tags through TagSpecifications on RunInstances/CreateFleet/ CreateLaunchTemplate, which AWS authorizes as CreateTags against the resource being created — a resource whose ID does not exist yet, so it cannot be named in the policy.

• The SSM parameter path is scoped to this workflow. That is the one place the policy is genuinely tight, and it is the important one: the parameters are the whole control channel, so a bastion that could read another workflow’s path could read its job commands.

parsl_ephemeral_provider.utils.aws.build_fleet_launch_template_configs(template_id: str, template_version: str, instance_types: List[str], subnet_id: str) → List[Dict[str, Any]]

Build the LaunchTemplateConfigs for a CreateFleet request (#86).

One config referencing one template, with an override per instance type so a single template still covers every pool the fleet may draw from.

CreateFleet’s override shape is richer than Spot Fleet’s – it also accepts ImageId, MaxPrice, and BlockDeviceMappings – but it still has no UserData, which is why the per-block user data has to live in the template rather than here.

Parameters

template_idstr

Launch template to draw the baseline definition from.

template_versionstr

Pinned version. Not $Latest: a fleet must launch the definition the caller built, not whatever a concurrent provider added afterwards.

instance_typesList[str]

Types to emit as overrides, in preference order.

subnet_idstr

Subnet every override launches into.

Returns

List[Dict[str, Any]]

A single-element LaunchTemplateConfigs list.

parsl_ephemeral_provider.utils.aws.build_launch_template_data(image_id: str, instance_type: str, subnet_id: str | None = None, security_group_id: str | None = None, associate_public_ip: bool = True, key_name: str | None = None, iam_instance_profile_arn: str | None = None, shutdown_behavior: str = 'terminate', user_data: str | None = None, monitoring: bool = False) → Dict[str, Any]

Build the LaunchTemplateData shared by every launch path (#85).

One definition serves the on-demand, spot, and fleet paths, which is what makes the Phase 6 EC2 Fleet/ASG migration possible – those APIs accept a template reference and nothing resembling RunInstances kwargs.

Every field is optional except the image and instance type, because the template is a baseline: RunInstances overrides UserData and TagSpecifications per launch, and Spot Fleet overrides InstanceType and SubnetId per pool.

Parameters

image_idstr

AMI to launch.

instance_typestr

Default instance type; fleet paths override it per pool.

subnet_idOptional[str]

Subnet for the primary network interface.

security_group_idOptional[str]

Security group for the primary network interface.

associate_public_ipbool

Whether the primary interface gets a public IP.

key_nameOptional[str]

EC2 key pair for SSH access.

iam_instance_profile_arnOptional[str]

Instance profile ARN; required for SSM command dispatch.

shutdown_behaviorstr

"terminate" or "stop" for an instance-initiated shutdown.

user_dataOptional[str]

Plaintext user data; base64-encoded here, since CreateLaunchTemplate does not do it.

monitoringbool

Whether to enable detailed CloudWatch monitoring.

Returns

Dict[str, Any]

A LaunchTemplateData document.

parsl_ephemeral_provider.utils.aws.create_bastion_instance_profile(session: Session, name_suffix: str, workflow_id: str) → str

Create (or reuse) an instance profile the bastion manager can work with.

The direct RunInstances bastion path had no profile at all (#229), so parsl-bastion-manager.py raised NoCredentialsError on its first AWS call and, under Restart=always, crash-looped every ten seconds. Nothing the bastion exists to do — polling for jobs, launching workers, terminating them — had ever worked on that path.

AmazonSSMManagedInstanceCore comes as a managed policy; the rest is inline because it is scoped to one workflow’s parameter path and so is not reusable.

Parameters

sessionboto3.Session

AWS session used to build the IAM and STS clients.

name_suffixstr

Discriminator for the role and profile names — the provider ID, so the pair is traceable to its provider and to the state that owns it.

workflow_idstr

Scopes the SSM parameter statement. The bastion reads and writes only its own workflow’s parameters.

Returns

str

ARN of the instance profile, ready to pass as IamInstanceProfile.

Raises

ResourceCreationError

If the role or profile cannot be created or retrieved.

parsl_ephemeral_provider.utils.aws.create_ec2_fleet(ec2_client: Any, launch_template_configs: List[Dict[str, Any]], target_capacity: int, allocation_strategy: str, client_token: str | None = None, tags: List[Dict[str, str]] | None = None, max_total_price: str | None = None) → Tuple[str, List[str]]

Create an instant EC2 Fleet and return its ID and instance IDs.

Replaces RequestSpotFleet, which AWS describes as “a legacy API with no planned investment” (#86).

Fleet type instant is deliberate: it places a synchronous request and returns the launched instance IDs in the response, so a block knows its instances without polling. That is what the rest of this package assumes, and the asynchronous types cannot provide it.

Several parameters the legacy path sent are rejected for this fleet type – verified against real EC2, with InvalidParameter rather than silent acceptance – so they are deliberately absent here:

• ReplaceUnhealthyInstances and TerminateInstancesWithExpiration (“not supported for given fleet type”)

• SpotOptions.MaintenanceStrategies, i.e. Capacity Rebalance (“only compatible with fleet type maintain”)

Parameters

ec2_clientAny

A boto3 EC2 client.

launch_template_configsList[Dict[str, Any]]

As returned by build_fleet_launch_template_configs().

target_capacityint

Number of instances to request, all spot.

allocation_strategystr

Spot allocation strategy; normalised to the kebab-case spelling CreateFleet requires.

client_tokenOptional[str]

Idempotency token. EC2 generates one when omitted.

tagsOptional[List[Dict[str, str]]]

Applied to both the fleet resource and the instances it launches, so either can be found by the cleanup sweep.

max_total_priceOptional[str]

Maximum hourly spot price for the whole fleet. Left unset by default: AWS warns that capping the price increases interruptions.

Returns

Tuple[str, List[str]]

The fleet ID, and the IDs of the instances it launched. The instance list may be shorter than target_capacity, or empty, if EC2 could not fill the request; the caller decides whether that is fatal.

Raises

ClientError

Propagated unchanged from CreateFleet, so the caller can discriminate on the EC2 error code. See the note at the call itself.

ValueError

If allocation_strategy is not one EC2 recognises.

parsl_ephemeral_provider.utils.aws.create_launch_template(ec2_client: Any, name: str, launch_template_data: Dict[str, Any], tags: List[Dict[str, str]] | None = None) → Tuple[str, str]

Create a launch template, reusing one that already exists under name.

Idempotent because initialize() may run again after a partial failure, and a second CreateLaunchTemplate under the same name is rejected with InvalidLaunchTemplateName.AlreadyExistsException. Rather than fail, the existing template is adopted – its name encodes the provider ID, so it can only be one this provider made.

Parameters

ec2_clientAny

A boto3 EC2 client.

namestr

Launch template name; must be unique within the account and region.

launch_template_dataDict[str, Any]

As returned by build_launch_template_data().

tagsOptional[List[Dict[str, str]]]

Tags applied to the template resource itself, for cleanup tracking.

Returns

Tuple[str, str]

The template ID and version number, the latter as the string RunInstances expects.

Raises

ResourceCreationError

If the template can neither be created nor found.

parsl_ephemeral_provider.utils.aws.create_session(region: str | None = None, profile_name: str | None = None, aws_access_key_id: str | None = None, aws_secret_access_key: str | None = None, aws_session_token: str | None = None, endpoint_url: str | None = None) → Session

Create a boto3 session with the given parameters.

Parameters

regionOptional[str], optional

AWS region to use, by default None

profile_nameOptional[str], optional

AWS profile name to use, by default None

aws_access_key_idOptional[str], optional

AWS access key ID, by default None

aws_secret_access_keyOptional[str], optional

AWS secret access key, by default None

aws_session_tokenOptional[str], optional

AWS session token, by default None

endpoint_urlOptional[str], optional

Custom endpoint URL for every AWS service, by default None. Use for VPC or FIPS endpoints, or to point the whole package at an emulator.

Returns

boto3.Session

The created boto3 session

Raises

AWSAuthenticationError

If authentication fails

AWSConnectionError

If connection to AWS services fails

parsl_ephemeral_provider.utils.aws.create_spot_interruption_notifier(events_client: Any, sqs_client: Any, name: str, tags: List[Dict[str, str]] | None = None) → Tuple[str, str, str]

Wire an EventBridge spot-interruption rule to a fresh SQS queue (#86).

This is what supplies the two-minute advance warning. Polling EC2 state cannot: an interrupted instance is only observable once it reaches shutting-down, which is after the reclaim, far too late to stop dispatching into it. An instant fleet also gets no Capacity Rebalance – CreateFleet rejects SpotOptions.MaintenanceStrategies for the type – so EventBridge is the mechanism, and it has the further advantage of working for instances that are already running.

No IAM role is created, and none is needed. Verified against real EventBridge: put_targets with an SQS ARN and no RoleArn returned FailedEntryCount=0. Unlike most target types, delivery to SQS is authorised by the queue’s resource policy, which is why this sets one granting events.amazonaws.com permission to sqs:SendMessage, conditioned on aws:SourceArn being this rule – so no other rule, in this account or any other, can post to the queue.

Verified end to end with a Fault Injection Simulator experiment (aws:ec2:send-spot-instance-interruptions): the warning reached the queue 15.2s after the experiment started, while the instance was still running.

Parameters

events_clientAny

A boto3 EventBridge client.

sqs_clientAny

A boto3 SQS client.

namestr

Name for both the rule and the queue. Must be unique per provider.

tagsOptional[List[Dict[str, str]]]

Applied to the rule so a leaked one is traceable. Not applied to the queue: SQS takes tags as a flat mapping, and the queue is named after the same provider anyway.

Returns

Tuple[str, str, str]

The rule name, the queue URL, and the queue ARN.

Raises

ResourceCreationError

If the rule, the queue, or the wiring between them cannot be created.

parsl_ephemeral_provider.utils.aws.create_tags(resource_ids: str | List[str], tags: Dict[str, str], session: Session, region: str | None = None) → None

Create tags for AWS resources.

Parameters

resource_idsUnion[str, List[str]]

Resource ID or list of resource IDs to tag

tagsDict[str, str]

Tags to apply to the resources

sessionboto3.Session

Boto3 session to use

regionOptional[str], optional

AWS region, by default None

Raises

ResourceCreationError

If tagging fails

parsl_ephemeral_provider.utils.aws.delete_bastion_instance_profile(session: Session, name_suffix: str) → bool

Delete the bastion role and instance profile named for name_suffix.

The inverse of create_bastion_instance_profile. Only for a pair this provider created — DetachedMode._owns_bastion_profile is the gate.

parsl_ephemeral_provider.utils.aws.delete_ec2_fleet(ec2_client: Any, fleet_id: str) → None

Delete an EC2 Fleet, terminating its instances.

Instance termination is not optional for an instant fleet: AWS rejects NoTerminateInstances for this type, and “a deleted instant fleet with running instances is not supported”.

Tolerates a fleet that is already gone, and a repeat call on one already deleting – verified that deleting twice succeeds both times, reporting deleted_terminating, and that an unknown ID comes back as a fleetIdDoesNotExist entry in UnsuccessfulFleetDeletions rather than as a raised error. Both are the desired outcome for a cleanup path, so neither raises.

Parameters

ec2_clientAny

A boto3 EC2 client.

fleet_idstr

Fleet to delete.

Raises

ResourceDeletionError

If EC2 refuses the deletion for any reason other than the fleet not existing.

parsl_ephemeral_provider.utils.aws.delete_instance_profile_pair(session: Session, profile_name: str, role_name: str) → bool

Delete instance profile profile_name and role role_name.

The inverse of _ensure_instance_profile plus its role. Only call this for a pair this provider created — StandardMode._owns_instance_profile and DetachedMode._owns_bastion_profile are the two gates. A caller-supplied profile belongs to the caller, and deleting it would break every other workload using it.

IAM enforces the teardown order: a profile holding a role cannot be deleted, and a role that is still in a profile or has policies attached cannot be deleted either. So it goes role-out-of-profile, profile, policies-off-role, role. Getting this wrong yields DeleteConflict rather than a partial success, which is why the order is not a matter of taste. It is also the order substrate’s CloudFormation gets wrong — it deletes the profile while the role is still in it — so this is worth keeping in one place.

Both attachment kinds are stripped. delete_role refuses while either a managed policy or an inline policy remains, and the two are listed and removed by different API calls; the bastion role carries an inline policy while the SSM worker role carries a managed one, so handling only one kind would strand whichever role used the other.

Every step tolerates NoSuchEntity: cleanup runs on paths that may already have partially completed, and a missing resource is the desired end state rather than an error.

Parameters

sessionboto3.Session

AWS session used to build the IAM client.

profile_namestr

Name of the instance profile to delete.

role_namestr

Name of the role to remove from it and then delete.

Returns

bool

True if the pair is gone (including “was already gone”), False if something could not be deleted. Never raises: a cleanup failure must not mask whatever the caller was originally doing.

parsl_ephemeral_provider.utils.aws.delete_launch_template(ec2_client: Any, template_id: str) → None

Delete a launch template, tolerating one that is already gone.

Deleting the template does not affect instances launched from it, so this is safe to call before those instances have terminated.

Parameters

ec2_clientAny

A boto3 EC2 client.

template_idstr

ID of the template to delete.

parsl_ephemeral_provider.utils.aws.delete_resource(resource_id: str, session: Session, resource_type: str, region: str | None = None, force: bool = False) → bool

Delete an AWS resource.

Parameters

resource_idstr

Resource ID to delete

sessionboto3.Session

Boto3 session to use

resource_typestr

Type of resource to delete

regionOptional[str], optional

AWS region, by default None

forcebool, optional

Whether to force deletion even if resource is in use, by default False

Returns

bool

True if the resource was deleted, False otherwise

Raises

ResourceDeletionError

If deletion fails

ResourceNotFoundError

If the resource is not found

parsl_ephemeral_provider.utils.aws.delete_spot_interruption_notifier(events_client: Any, sqs_client: Any, rule_name: str | None, queue_url: str | None) → None

Tear down what create_spot_interruption_notifier() built.

Order matters: the target has to go before the rule, because EventBridge refuses to delete a rule that still has one. Every step logs rather than raises – this runs from cleanup paths, where a rule that is already gone is the desired end state and must not stop the queue from being deleted too.

Parameters

events_clientAny

A boto3 EventBridge client.

sqs_clientAny

A boto3 SQS client.

rule_nameOptional[str]

Rule to delete. Ignored when None.

queue_urlOptional[str]

Queue to delete. Ignored when None.

parsl_ephemeral_provider.utils.aws.delete_ssm_instance_profile(session: Session, name_suffix: str) → bool

Delete the SSM role and instance profile named for name_suffix.

The inverse of get_or_create_ssm_instance_profile. Only call this for a pair this provider created — see StandardMode._owns_instance_profile. A caller-supplied profile belongs to the caller, and deleting it would break every other workload using it.

Parameters

sessionboto3.Session

AWS session used to build the IAM client.

name_suffixstr

The same discriminator passed to get_or_create_ssm_instance_profile — normally the provider ID.

Returns

bool

True if the pair is gone (including “was already gone”), False if something could not be deleted.

parsl_ephemeral_provider.utils.aws.describe_ec2_fleet(ec2_client: Any, fleet_id: str) → Dict[str, Any] | None

Return the fleet’s FleetData, or None if EC2 has forgotten it.

The ID is always passed explicitly, and not merely as an optimisation: AWS documents that “if a fleet is of type instant, you must specify the fleet ID in the request, otherwise the fleet does not appear in the response”. Verified – describe_fleets() with no FleetIds returned an empty list while an instant fleet was active.

Parameters

ec2_clientAny

A boto3 EC2 client.

fleet_idstr

Fleet to describe.

Returns

Optional[Dict[str, Any]]

The FleetData document, or None when the fleet no longer exists.

parsl_ephemeral_provider.utils.aws.describe_instance_capacity(session: Session, instance_type: str) → Tuple[int | None, float | None]

Look up an instance type’s vCPU count and memory in GB.

Parsl’s ExecutionProvider declares cores_per_node and mem_per_node so an executor can size its worker pool: HTEX divides them by its per-worker requirements to pick workers_per_node, and falls back to a hardcoded guess of 1 when both are None. EC2 already knows the real numbers, so there is no reason to make the caller supply them.

Failure is not an error. This is an optimisation hint, and a provider that cannot reach EC2 during __init__ should still construct — so any exception yields (None, None), which is exactly the base class’s default.

Parameters

sessionboto3.Session

Session used to call ec2:DescribeInstanceTypes.

instance_typestr

Instance type to describe, e.g. "t3.micro".

Returns

Tuple[Optional[int], Optional[float]]

(vcpus, memory_gb), or (None, None) if the lookup failed.

parsl_ephemeral_provider.utils.aws.encode_user_data(user_data: str) → str

Base64-encode user_data for an API that does not do it for you.

botocore installs base64_encode_user_data on before-parameter-build.ec2.RunInstances only – verified by inspecting botocore.handlers.BUILTIN_HANDLERS, where that operation and autoscaling.CreateLaunchConfiguration are the sole entries. CreateLaunchTemplate is not among them, so a plaintext script passed there is stored verbatim, handed to cloud-init base64-decoded, and produces garbage that fails silently – the instance boots fine and simply never runs the worker.

Encoding twice is the other half of the trap, so callers must not hand already-encoded data to RunInstances.

Parameters

user_datastr

The plaintext user data script.

Returns

str

The base64-encoded form.

parsl_ephemeral_provider.utils.aws.get_cf_template(template_name: str) → str

Load a CloudFormation template from the templates directory.

Uses importlib.resources so the template is found both in an installed wheel and in an editable/source checkout. The previous implementation called pkg_resources.resource_string with the import placed outside its own try — so once setuptools 81 removed pkg_resources the except ModuleNotFoundError fallback became unreachable and every call raised, taking down DetachedMode.initialize() with it. It also fell back to a placeholder template declaring no Outputs, which the bastion path then indexed for BastionHostId — a confusing failure several steps removed from the missing file (#112).

Parameters

template_namestr

Name of the template file (e.g., ‘bastion.yml’)

Returns

str

CloudFormation template content

Raises

FileNotFoundError

If the template is not found in the package or on the filesystem

parsl_ephemeral_provider.utils.aws.get_default_ami(region: str, architecture: str = 'x86_64', session: Session | None = None) → str

Resolve the latest Amazon Linux 2023 AMI for a region and architecture.

Resolution order:

1. AWS’s public SSM Parameter Store alias /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-<arch>, which AWS repoints at every new AL2023 release. This is the only source that stays correct without maintenance.

2. DEFAULT_AMI_MAPPING, retained purely so offline test runs against moto or substrate need no network. It is not a reliable source of live AMIs – see the note in constants.py – and is x86_64-only, so it is skipped for arm64 rather than returning an image that cannot boot.

Parameters

regionstr

AWS region.

architecturestr, optional

"x86_64" (default) or "arm64". Use architecture_for_instance_type() to derive it from an instance type.

sessionboto3.Session, optional

Session to query SSM with. A new one is created for region when omitted; pass an existing session to reuse its credentials and any custom endpoint.

Returns

str

AMI ID.

Raises

AMINotFoundError

If SSM cannot be reached and no usable fallback exists for the region and architecture.

parsl_ephemeral_provider.utils.aws.get_ec2_fleet_instance_ids(ec2_client: Any, fleet_id: str) → List[str]

Return the IDs of instances belonging to fleet_id.

Goes through describe_instances filtered on the aws:ec2:fleet-id tag EC2 applies to every fleet-launched instance, because the two obvious routes do not work for an instant fleet – verified against real EC2:

• describe_fleet_instances refuses it outright with Unsupported: “Describe fleet instances is not supported by this type of fleet.”

• describe_fleets does return an Instances list, but only reflects the original launch; it does not drop instances that have since terminated.

Terminated instances are excluded, so the result answers “what is this fleet still running” rather than “what did it ever launch”.

Parameters

ec2_clientAny

A boto3 EC2 client.

fleet_idstr

Fleet whose instances to list.

Returns

List[str]

Instance IDs that are not terminated. Empty if the fleet launched nothing, or everything it launched is gone.

parsl_ephemeral_provider.utils.aws.get_or_create_iam_role(iam_client: Any, role_name: str, assume_role_policy: Dict[str, Any], policy_arns: List[str], tags: List[Dict[str, str]] | None = None, description: str = '') → str

Get or create an IAM role idempotently.

Checks whether the role already exists and returns its ARN without modifying it. If the role does not exist, creates it, attaches the supplied managed-policy ARNs, and returns the new ARN.

Parameters

iam_clientAny

A boto3 IAM client.

role_namestr

Name of the IAM role.

assume_role_policyDict[str, Any]

Trust-relationship policy document (Python dict, not JSON string).

policy_arnsList[str]

Managed policy ARNs to attach.

tagsOptional[List[Dict[str, str]]], optional

Tags to apply when creating the role.

descriptionstr, optional

Role description.

Returns

str

ARN of the existing or newly created role.

Raises

ResourceCreationError

If role creation or retrieval fails.

parsl_ephemeral_provider.utils.aws.get_or_create_ssm_instance_profile(session: Session, name_suffix: str, iam_instance_profile_arn: str | None = None, auto_create: bool = False) → str | None

Resolve an IAM instance profile ARN granting SSM access, or None.

SSM SendCommand — used by warm-pool and one-shot dispatch — needs the instance to carry a profile with the AmazonSSMManagedInstanceCore policy. Without it the SSM agent never registers and every command dispatch times out.

Resolution order:

1. iam_instance_profile_arn if supplied → used directly.

2. auto_create → get-or-create a profile holding AmazonSSMManagedInstanceCore.

3. Otherwise → None; the caller launches instances without a profile.

Both the create and the attach steps are idempotent, so concurrent callers sharing a name_suffix converge on the same profile.

Parameters

sessionboto3.Session

AWS session used to build the IAM client.

name_suffixstr

Discriminator appended to the role and profile names — normally the provider ID, so resources are traceable back to their provider.

iam_instance_profile_arnOptional[str], optional

Pre-existing profile ARN to use verbatim, by default None.

auto_createbool, optional

Whether to create a profile when none was supplied, by default False.

Returns

Optional[str]

ARN of the resolved instance profile, or None when neither an explicit ARN was given nor auto-creation requested.

Raises

ResourceCreationError

If the profile cannot be created or retrieved.

parsl_ephemeral_provider.utils.aws.get_resources_by_tags(tags: Dict[str, str], session: Session, region: str | None = None, resource_type: str | None = None) → List[Dict[str, Any]]

Get AWS resources by tags.

Parameters

tagsDict[str, str]

Tags to filter resources by

sessionboto3.Session

Boto3 session to use

regionOptional[str], optional

AWS region, by default None

resource_typeOptional[str], optional

Resource type to filter by, by default None

Returns

List[Dict[str, Any]]

List of resources matching the tags

Raises

AWSConnectionError

If connection to AWS services fails

parsl_ephemeral_provider.utils.aws.normalize_ec2_fleet_allocation_strategy(strategy: str) → str

Translate an allocation strategy to the spelling CreateFleet takes.

The mirror image of normalize_spot_fleet_allocation_strategy(): CreateFleet accepts only kebab-case, and rejects the camelCase spelling RequestSpotFleet demands. The provider’s spot_allocation_strategy kwarg is documented in kebab-case, so the common case is a pass-through – but a caller who supplied the camelCase form (or read it off the legacy constant) is converted rather than punished.

Validating here matters more than it does on the legacy path, because EC2 does not catch this for you until real capacity is requested: verified against real EC2 that both DryRun=True and TotalTargetCapacity=0 accept "priceCapacityOptimized", and describe_fleets then shows the bad value stored verbatim.

Parameters

strategystr

Allocation strategy in either spelling.

Returns

str

The kebab-case spelling CreateFleet accepts.

Raises

ValueError

If the strategy is not one EC2 recognises, or is not a string.

parsl_ephemeral_provider.utils.aws.normalize_spot_fleet_allocation_strategy(strategy: str) → str

Translate an allocation strategy to the spelling RequestSpotFleet takes.

The two fleet APIs disagree on the casing of the same enum, and each rejects the other’s spelling. Verified against real EC2 in us-east-1 – RequestSpotFleet with "price-capacity-optimized" returns InvalidParameterValue, and CreateFleet with "priceCapacityOptimized" returns InvalidParameter.

The provider’s spot_allocation_strategy kwarg is documented in kebab-case, so it needs converting at the RequestSpotFleet boundary. Values already in camelCase pass through, so a caller who supplied the API-native spelling is not punished for it.

Parameters

strategystr

Allocation strategy in either spelling, e.g. "price-capacity-optimized" or "priceCapacityOptimized".

Returns

str

The camelCase spelling RequestSpotFleet accepts.

Raises

ValueError

If the strategy is not one EC2 recognises, or is not a string. Raised here rather than letting EC2 reject it, because a spot fleet request fails several seconds and one IAM role later.

parsl_ephemeral_provider.utils.aws.resolve_manager_session(provider: Any, credential_manager: Any) → Session

Return the session a compute manager should use.

The caller’s own session wins. Only when the provider has none does this fall back to building one from the credential manager.

All four compute managers previously went straight to credential_manager.create_boto3_session(), discarding provider.session entirely — so a caller who passed an explicitly configured session (temporary role credentials, a chosen profile, a LocalStack endpoint_url) had it silently replaced by one built from ambient environment credentials, possibly pointing at a different account (#117). It also meant an injected test double was ignored and the manager reached real AWS; a unit test created a live ECS cluster this way.

This mirrors the fix already applied to the state stores, which had the same defect.

Parameters

providerAny

The provider (or operating mode) the manager was constructed with.

credential_managerAny

Fallback used when the provider carries no session.

Returns

boto3.Session

The caller’s session if it has one, else a newly built session.

parsl_ephemeral_provider.utils.aws.ssm_instance_profile_names(name_suffix: str) → Tuple[str, str]

Return the (role_name, profile_name) pair for name_suffix.

The names are derived, not stored, so the creator and the deleter cannot drift apart. get_or_create_ssm_instance_profile and delete_ssm_instance_profile are the two callers.

parsl_ephemeral_provider.utils.aws.wait_for_resource(resource_id: str, waiter_name: str, service_client: Any, waiter_config: Dict[str, Any] | None = None, resource_name: str = 'resource', delay: int = 5, max_attempts: int = 60) → None

Wait for a resource to reach the desired state.

Parameters

resource_idstr

Resource ID to wait for

waiter_namestr

Name of the waiter to use

service_clientAny

Boto3 service client

waiter_configOptional[Dict[str, Any]], optional

Waiter configuration, by default None

resource_namestr, optional

Name of the resource for logging purposes, by default “resource”

delayint, optional

Seconds between waiter attempts, by default 5

max_attemptsint, optional

Maximum number of waiter attempts, by default 60

Raises

ResourceCreationError

If the resource fails to reach the desired state