API Reference - Executors

Core Executor Classes

DefaultExecutor

class DefaultExecutor

The simplest executor that runs tasks synchronously in the current thread.

Methods:

submit(fn, /, *args, **kwargs) → Future

Executes the function immediately in the current thread.

Parameters:

  • fn – Function to execute

  • args – Positional arguments

  • kwargs – Keyword arguments

Returns:

Future object with the result

Return type:

Future

ThreadPoolExecutor & ProcessPoolExecutor

Standard Python concurrent.futures executors for parallel execution.

Configuration:

  • max_workers: Number of worker threads/processes

  • thread_name_prefix: Prefix for worker thread names (ThreadPoolExecutor only)

  • max_tasks_per_child: Maximum tasks per worker process (ProcessPoolExecutor only)

Remote Executors

RemoteExecutor

class RemoteExecutor

Executes tasks on remote servers using raw socket connections.

Parameters:

Parameters:

  • host – Remote server hostname/IP

  • port – Remote server port

  • timeout – Connection timeout in seconds

  • use_encryption – Whether to use SSL/TLS encryption

  • client_cert_path – Path to client certificate file

  • client_key_path – Path to client private key file

  • ca_cert_path – Path to CA certificate file

Methods:

submit(fn, /, *args, **kwargs) → Future

Submit task for remote execution.

Parameters:

  • fn – Function to execute

  • args – Positional arguments

  • kwargs – Keyword arguments

Returns:

Future representing the remote execution

Return type:

Future

XMLRPCExecutor

class XMLRPCExecutor

Executes tasks on remote servers using XML-RPC protocol.

Parameters:

Parameters:

  • host – Remote server hostname/IP

  • port – Remote server port

  • max_workers – Maximum number of worker threads

  • use_encryption – Whether to use SSL/TLS

  • client_cert_path – Path to client certificate

  • client_key_path – Path to client key

  • ca_cert_path – Path to CA certificate

GRPCExecutor

class GRPCExecutor

Executes tasks on remote servers using gRPC protocol.

Parameters:

Parameters:

  • host – Remote server hostname/IP

  • port – Remote server port

  • max_workers – Maximum number of worker threads

  • use_encryption – Whether to use SSL/TLS

  • client_cert_path – Path to client certificate

  • client_key_path – Path to client key

  • ca_cert_path – Path to CA certificate

Distributed Executors

HadoopExecutor

class HadoopExecutor

Executes tasks on a Hadoop cluster using MapReduce.

Parameters:

Parameters:

  • host – Hadoop namenode hostname

  • port – Hadoop namenode port

  • username – Username for authentication

  • password – Password for authentication

  • kerb_ticket – Path to Kerberos ticket file

  • hdfs_config – Additional HDFS configuration

  • yarn_config – Additional YARN configuration

  • max_workers – Maximum concurrent workers

  • poll_interval – Job status polling interval

Methods:

submit(fn, /, *args, **kwargs) → Future

Submit task for Hadoop execution.

Parameters:

  • fn – Function to execute (must be EventBase instance)

  • args – Positional arguments

  • kwargs – Keyword arguments

Returns:

Future representing the Hadoop job

Return type:

Future

submit_batch(fns: List[Callable], *args_list) → List[Future]

Submit multiple tasks as a batch.

Parameters:

  • fns – List of functions to execute

  • args_list – Arguments for each function

Returns:

List of Futures

Return type:

List[Future]

cancel(future: Future) → bool

Cancel a submitted job.

Parameters:

future – Future representing the job

Returns:

True if cancellation successful

Return type:

bool