API

MetaData

class ska_sdp_dataproduct_metadata.metadata.MetaData(path=None)

Class for generating the metadata file

Parameters:: path – location of the metadata file to read

exception ValidationError(message, errors): An exception indicating an error during validation of metadata against the schema.

get_data(): Return the data dictionary within the MetaData object

load_processing_block(pb_id=None, mount_path=None)

Configure a MetaData object based on the data in a processing block

Parameters:: pb_id – processing block ID

metadata_schema = <_io.TextIOWrapper name='/home/docs/checkouts/readthedocs.org/user_builds/sdp-data-product-metadata/checkouts/latest/src/ska_sdp_dataproduct_metadata/schema/metadata.json' mode='r' encoding='utf-8'>

new_file(dp_path=None, description=None, crc=None)

Creates a new file into the metadata and add current file status.

Parameters:

dp_path – path of the data product Not to be confused with path of the metadata file
description – Description of the file
crc – CRC (Cyclic Redundancy Check) checksum for the file. NB: CRC is supplied, not calculated

Returns:

instance of the File class

property output_path: Output metadata path

read(file)

Read input metadata file and load in yaml.

Parameters:: file – input metadata file
Returns:: Returns the yaml loaded metadata file

runtime_abspath(path)

The absolute path of path relative to the standard prefix. This value is valid at runtime; i.e., it maps to the filesystem in use.

Parameters:: path – A path relative to the standard prefix.

set_config(script)

Set configuration of generating software.

Parameters:: script – Processing script details

set_execution_block_id(execution_block_id)

Set the execution_block_id for this MetaData object NB: If this MetaData object describes a dataproduct that was not generated from an execution_block, then it is possible to use any SKA Unique Identifier (https://gitlab.com/ska-telescope/ska-ser-skuid)

Parameters:: execution_block_id – an execution_block_id

validate() → list

Validate the current contents of the metadata against the schema.

Returns:: A list of errors.

validator = Draft202012Validator(schema={'additionalProperties': True, 'properties': {'config': {'additionalProperties': True, 'properties': {'cmdline': {'type': ['string', 'null']}, 'commit': {'type': ['string', 'null']}, 'image': {'type': ['string', 'null']}, 'processing_block': {'type': ['string', 'null']}, ...}, 'required': [], 'type': 'object'}, 'context': {'additionalProperties': True, 'properties': {'intent': {'type': 'string'}, 'notes': {'type': 'string'}, 'observer': {'type': 'string'}}, 'required': [], 'type': 'object'}, 'execution_block': {'type': 'string'}, 'files': {'items': {'additionalProperties': True, 'properties': {'crc': {'type': [...]}, 'description': {'type': 'string'}, 'path': {'type': 'string'}, 'size': {'type': 'integer'}, ...}, 'required': [], 'type': 'object'}, 'type': 'array'}, ...}, 'required': ['execution_block'], 'type': 'object'}, format_checker=None)

write(): Write the metadata to a yaml file.

File

class ska_sdp_dataproduct_metadata.metadata.File(metadata, path)

Class to represent the file in the metadata.

property full_path: Get the full path object.

update_status(status)

Update the current file status.

Param:: status: status to be updated to

ObsCore

class ska_sdp_dataproduct_metadata.obscore.ObsCore

SKA-specific possible values for ObsCore attributes

class AccessFormat(value)

The format (mime-type) of the data product if downloaded as a file

BINARY = 'application/octet-stream'

FITS = 'image/fits'

HDF5 = 'application/x-hdf5'

JPEG = 'image/jpeg'

PNG = 'image/png'

TAR_GZ = 'application/x-tar-gzip'

UNKNOWN = 'application/unknown'

class CalibrationLevel(value)

The amount of calibration processing that has been applied to create the data product Refer to the IVOA standard for a full description of the categories

LEVEL_0 = 0

LEVEL_1 = 1

LEVEL_2 = 2

LEVEL_3 = 3

LEVEL_4 = 4

class DataProductType(value)

A simple string value describing the primary nature of the data product

MS = 'MS'

POINTING = 'POINTING-OFFSETS'

UNKNOWN = 'Unknown'

class ObservationCollection(value)

A string identifying the data collection to which the data product belongs

SIMULATION = 'Simulation'

UNKNOWN = 'Unknown'

SKA = 'SKA-Observatory'

SKA_LOW = 'SKA-LOW'

SKA_MID = 'SKA-MID'

class UCD(value)

A list of Unified Content Descriptors (Preite Martinez, et al. 2007) describing the nature of the observable within the data product https://www.ivoa.net/documents/latest/UCDlist.html

COUNT = 'phot.count'

FLUX_DENSITY = 'phot.flux.density'

FOURIER = 'stat.fourier'

UNKNOWN = 'Unknown'