API Reference - Fields and Signals

Field Classes

InputDataField

class InputDataField

Base field class for defining pipeline input data.

Parameters:

Parameters:

  • name (Optional[str]) – Field name

  • required (bool) – Whether field is required

  • data_type (Union[Type, Tuple[Type]]) – Expected data type(s)

  • default (Any) – Default value

  • batch_processor (Optional[Callable]) – Function for batch processing

  • batch_size (int) – Size of batches for processing

Properties:

property has_batch_operation: bool

Whether field has batch processing configured.

FileInputDataField

class FileInputDataField

Field class for handling file inputs in pipelines.

Parameters:

Parameters:

  • path (Union[str, PathLike]) – File path

  • required (bool) – Whether file is required

  • chunk_size (int) – Size of chunks when reading file

  • mode (str) – File open mode

  • encoding (Optional[str]) – File encoding

Methods:

Inherits all methods from InputDataField and adds file handling capabilities.

Signal Classes

SoftSignal

class SoftSignal

Implementation of the soft signaling system.

Parameters:

Parameters:

  • name (str) – Signal name

  • provide_args (Optional[List[str]]) – Arguments provided by signal

Methods:

connect(sender: Any, listener: Callable) → None

Connects a listener function to the signal.

Parameters:

  • sender – Object sending the signal

  • listener – Callback function

disconnect(sender: Any, listener: Callable) → None

Disconnects a listener from the signal.

Parameters:

  • sender – Object sending the signal

  • listener – Callback function to disconnect

emit(sender: Any, **kwargs) → List[Tuple[Any, Any]]

Emits the signal to all connected listeners.

Parameters:

  • sender – Object sending the signal

  • kwargs – Additional arguments for listeners

Returns:

List of (receiver, response) tuples

Built-in Signals

Pipeline Lifecycle

pipeline_pre_init: SoftSignal

Emitted before pipeline initialization. Arguments: cls, args, kwargs

pipeline_post_init: SoftSignal

Emitted after pipeline initialization. Arguments: pipeline

pipeline_execution_start: SoftSignal

Emitted when pipeline execution starts. Arguments: pipeline

pipeline_execution_end: SoftSignal

Emitted when pipeline execution ends. Arguments: execution_context

Event Lifecycle

event_init: SoftSignal

Emitted when an event is initialized. Arguments: event, init_kwargs

event_execution_init: SoftSignal

Emitted before event execution. Arguments: event, execution_context, executor, call_kwargs

event_execution_start: SoftSignal

Emitted when event execution starts. Arguments: event, execution_context

event_execution_end: SoftSignal

Emitted when event execution completes. Arguments: event, execution_context, future