ska_sdp_piper.piper.pipeline module

class ska_sdp_piper.piper.pipeline.Pipeline(name, *stages, global_config_model=<class 'ska_sdp_piper.piper.piper_base_model.PiperBaseModel'>, version=None)

Bases: Command

property config: dict

Get all the configuration of the pipeline

This returns a dictionary with following top level keys:

• global_parameters

• parameters

• pipeline

• version

with_qa_path_resolver(resolver)

Register a custom function to resolve the log file path dynamically.

Parameters:

resolver (Callable) -- A function which can provide alternate log file paths.

Return type:

Pipeline

overide_run(*cli_args, runner=None)

Override the run subcommand with custom CLI arguments and runner.

Parameters:

• *cli_args (CLIArgument) -- List of additional CLI arguments to register

• runner (type[DefaultRunner] | None, default: None) -- The runner class to use for execution of stages

Return type:

Pipeline

Returns:

The current pipeline instance.

create_output_dir(output_path, unique_output_subdir, **kwargs)

Create the directory for pipeline output files.

Parameters:

• output_path (str) -- The root path for outputs.

• unique_output_subdir (bool) -- Whether to create a unique timestamped subdirectory.

• **kwargs (dict) -- Additional arguments for directory creation.

configure_logging(verbose, **kwargs)

Initialize logging for the pipeline run.

Parameters:

• verbose (int) -- The verbosity level (non-zero enables verbose logging).

• **kwargs (dict) -- Additional arguments for log configuration.

set_runtime(config_path, override_defaults, stages, **kwargs)

Set the runtime state based on CLI and configuration files.

Parameters:

• config_path (str) -- Path to the YAML configuration file.

• override_defaults (dict) -- Key-value pairs to override default settings.

• stages (list[str]) -- Specific stages to enable for this run.

• **kwargs (dict) -- Additional runtime parameters.

install_config(override_defaults, config_install_path, **kwargs)

Set the runtime state based on CLI and configuration files.

Parameters:

• override_defaults (list[tuple[str, str]]) -- A list of tuple/list, where each element contains 2 sub-elements. First element is a string key to the parameter to override. This can represent a path to a nested key, where each key in the hirerarchy is seperated by a . character. Second element is also a string, which is then parsed using YAML rules, and converted to a rich object.

• config_install_path (str) -- Directory path to store the default config

• **kwargs (dict) -- Additional runtime parameters.

write_runtime_params(cli_args)

Persist current configuration and CLI arguments to disk.

Parameters:

cli_args (dict) -- The raw CLI arguments used for the run.

resolve_qa_dir(**kwargs)

Determine and set the QA directory path.

Parameters:

**kwargs (dict) -- Arbitrary keyword arguments passed to the QA path resolver.

property executable_stages: list[Stage]

Get the list of stages currently enabled for execution.

Return type:

List of stage instances of all "enabled" stages.

run(**cli_args)

Execute the pipeline lifecycle.

Parameters:

**cli_args -- Arguments parsed from the command line.

Return type:

Any

Returns:

The result returned by the runner's execution.