dp3::steps::Step Class Reference

Abstract base class for a DP3 step. More...

#include <Step.h>

Inheritance diagram for dp3::steps::Step:

Public Types
enum class	MsType { kRegular , kBda }
	To check compatibility between steps before running. More...
typedef std::shared_ptr< Step >	ShPtr

Public Member Functions
	Step ()
virtual	~Step ()
virtual bool	accepts (MsType dt) const
	Boolean if this step can process this type of data. More...
virtual void	finish ()=0
	Finish the processing of this step and subsequent steps. More...
const base::DPInfo &	getInfoIn () const
	Get access to the info of the input. More...
const base::DPInfo &	getInfoOut () const
	Get access to the info of the output. More...
const Step::ShPtr &	getNextStep () const
	Get the next step. More...
Step *	getPrevStep () const
	Get the previous step. More...
virtual dp3::common::Fields	getProvidedFields () const =0
virtual dp3::common::Fields	getRequiredFields () const =0
	Get the fields required by the current step. More...
virtual MsType	outputs () const
	Return which datatype this step outputs. More...
virtual bool	process (std::unique_ptr< base::BdaBuffer >)
virtual bool	process (std::unique_ptr< base::DPBuffer > buffer)
void	setInfo (const base::DPInfo &)
virtual void	setNextStep (Step::ShPtr nextStep)
	Set the next step. More...
void	setPrevStep (Step *prevStep)
	Set the previous step. More...
virtual void	show (std::ostream &) const =0
	Show the step parameters. More...
virtual void	showCounts (std::ostream &) const
virtual void	showTimings (std::ostream &, double duration) const
virtual void	updateInfo (const base::DPInfo &)

Static Public Member Functions
static void	SetThreadingIsInitialized ()

Static Public Attributes
static constexpr dp3::common::Fields	kDataField
static constexpr dp3::common::Fields	kFlagsField
static constexpr dp3::common::Fields	kUvwField
static constexpr dp3::common::Fields	kWeightsField

Protected Member Functions
virtual void	addToMS (const std::string &msName)
base::DPInfo &	GetWritableInfoOut ()

Detailed Description

Abstract base class for a DP3 step.

This class defines a step in the DP3 pipeline. It is an abstract class from which all steps should be derived. A few functions can or must be implemented. They are called by the DP3 program in the following order.

• 'updateInfo' should update its DPInfo object with the specific step information. For example, in this way it is known in all steps how the data are averaged and what the shape is.
• 'show' can be used to show the attributes.
• 'process' is called continuously to process the next time slot. When processed, it should call 'process' of the next step. When done (i.e. at the end of the input), it should return False.
• 'finish' finishes the processing which could mean that 'process' of the next step has to be called several times. When done, it should call 'finish' of the next step.
• 'addToMS' is called in the 'finish' step of tasks that write or update a measurement set. It gives a step the opportunity to add some data to the MS written/updated. It is, for example, used by AOFlagger to write its statistics.
• 'showCounts' can be used to show possible counts of flags, etc.

A Step object contains a DPInfo object telling the data settings for a step (like channel info, baseline info, etc.).

Member Typedef Documentation

ShPtr

typedef std::shared_ptr<Step> dp3::steps::Step::ShPtr

Member Enumeration Documentation

MsType

enum dp3::steps::Step::MsType

strong

To check compatibility between steps before running.

Enumerator
kRegular
kBda

Constructor & Destructor Documentation

Step()

dp3::steps::Step::Step

(

)

~Step()

virtual dp3::steps::Step::~Step ( )

virtual

Member Function Documentation

accepts()

virtual bool dp3::steps::Step::accepts ( MsType  dt ) const

inlinevirtual

Boolean if this step can process this type of data.

Reimplemented in dp3::steps::NullStep, dp3::steps::BdaAverager, dp3::steps::UVWFlagger, dp3::steps::ScaleData, dp3::steps::Predict, dp3::steps::MsColumnReader, dp3::steps::BdaGroupPredict, dp3::steps::BdaExpander, and dp3::steps::BdaDdeCal.

addToMS()

virtual void dp3::steps::Step::addToMS ( const std::string &  msName )

protectedvirtual

Add some data to the MeasurementSet written/updated. The default implementation only calls addToMS from the previous step

Reimplemented in dp3::steps::StationAdder, dp3::steps::Filter, dp3::steps::Demixer, dp3::steps::AOFlaggerStep, and dp3::steps::MsReader.

finish()

virtual void dp3::steps::Step::finish ( )

pure virtual

Finish the processing of this step and subsequent steps.

Implemented in dp3::steps::WSCleanWriter, dp3::steps::WGridderPredict, dp3::steps::UVWFlagger, dp3::steps::Upsample, dp3::steps::test::ThrowStep, dp3::steps::MockStep, dp3::steps::MockInput, dp3::steps::SVPInput, dp3::steps::StationAdder, dp3::steps::Split, dp3::steps::SetBeam, dp3::steps::ScaleData, dp3::steps::SagecalPredict, dp3::steps::ResultStep, dp3::steps::PreFlagger, dp3::steps::Predict, dp3::steps::PhaseShift, dp3::steps::OnePredict, dp3::steps::OneApplyCal, dp3::steps::NullStokes, dp3::steps::NullStep, dp3::steps::MultiResultStep, dp3::steps::MultiMsReader, dp3::steps::MSWriter, dp3::steps::MSUpdater, dp3::steps::MsReader, dp3::steps::MsColumnReader, dp3::steps::MSBDAWriter, dp3::steps::MSBDAReader, dp3::steps::MadFlagger, dp3::steps::Interpolate, dp3::steps::IDGPredict, dp3::steps::IDGImager, dp3::steps::H5ParmPredict, dp3::steps::GainCal, dp3::steps::Filter, dp3::steps::DummyStep, dp3::steps::Demixer, dp3::steps::DDECal, dp3::steps::Counter, dp3::steps::Clipper, dp3::steps::BDAResultStep, dp3::steps::BdaGroupPredict, dp3::steps::BdaExpander, dp3::steps::BdaDdeCal, dp3::steps::BdaAverager, dp3::steps::Averager, dp3::steps::ApplyCal, dp3::steps::ApplyBeam, dp3::steps::AOFlaggerStep, dp3::steps::AntennaFlagger, dp3::pythondp3::PyStep, dp3::steps::Transfer, dp3::steps::FlagTransfer, dp3::steps::DynSpec, and dp3::steps::Combine.

getInfoIn()

const base::DPInfo& dp3::steps::Step::getInfoIn ( ) const

inline

Get access to the info of the input.

getInfoOut()

const base::DPInfo& dp3::steps::Step::getInfoOut ( ) const

inline

Get access to the info of the output.

getNextStep()

const Step::ShPtr& dp3::steps::Step::getNextStep ( ) const

inline

Get the next step.

getPrevStep()

Step* dp3::steps::Step::getPrevStep ( ) const

inline

Get the previous step.

getProvidedFields()

virtual dp3::common::Fields dp3::steps::Step::getProvidedFields ( ) const

pure virtual

Get the fields provided (modified and/or created) by the current step. The returned fields thus should not include (required) fields that are forwarded without modifications.

Implemented in dp3::steps::WGridderPredict, dp3::steps::UVWFlagger, dp3::steps::Upsample, dp3::steps::test::ThrowStep, dp3::steps::MockInput, dp3::steps::ModelDataStep, dp3::steps::StationAdder, dp3::steps::SetBeam, dp3::steps::ScaleData, dp3::steps::SagecalPredict, dp3::steps::ResultStep, dp3::steps::PreFlagger, dp3::steps::Predict, dp3::steps::PhaseShift, dp3::steps::OutputStep, dp3::steps::OnePredict, dp3::steps::OneApplyCal, dp3::steps::NullStokes, dp3::steps::NullStep, dp3::steps::MultiResultStep, dp3::steps::MsColumnReader, dp3::steps::MadFlagger, dp3::steps::Interpolate, dp3::steps::InputStep, dp3::steps::IDGImager, dp3::steps::H5ParmPredict, dp3::steps::GainCal, dp3::steps::Filter, dp3::steps::DummyStep, dp3::steps::Demixer, dp3::steps::DDECal, dp3::steps::Counter, dp3::steps::Clipper, dp3::steps::BDAResultStep, dp3::steps::BdaExpander, dp3::steps::BdaDdeCal, dp3::steps::BdaAverager, dp3::steps::Averager, dp3::steps::ApplyCal, dp3::steps::ApplyBeam, dp3::steps::AOFlaggerStep, dp3::steps::AntennaFlagger, dp3::pythondp3::PyStep, dp3::steps::Transfer, dp3::steps::FlagTransfer, dp3::steps::DynSpec, and dp3::steps::Combine.

getRequiredFields()

virtual dp3::common::Fields dp3::steps::Step::getRequiredFields ( ) const

pure virtual

Get the fields required by the current step.

Implemented in dp3::steps::WSCleanWriter, dp3::steps::WGridderPredict, dp3::steps::UVWFlagger, dp3::steps::Upsample, dp3::steps::test::ThrowStep, dp3::steps::MockInput, dp3::steps::StationAdder, dp3::steps::Split, dp3::steps::SetBeam, dp3::steps::ScaleData, dp3::steps::SagecalPredict, dp3::steps::ResultStep, dp3::steps::PreFlagger, dp3::steps::Predict, dp3::steps::PhaseShift, dp3::steps::OnePredict, dp3::steps::OneApplyCal, dp3::steps::NullStokes, dp3::steps::NullStep, dp3::steps::MultiResultStep, dp3::steps::MSWriter, dp3::steps::MSUpdater, dp3::steps::MsColumnReader, dp3::steps::MSBDAWriter, dp3::steps::MadFlagger, dp3::steps::Interpolate, dp3::steps::InputStep, dp3::steps::IDGPredict, dp3::steps::IDGImager, dp3::steps::H5ParmPredict, dp3::steps::GainCal, dp3::steps::Filter, dp3::steps::DummyStep, dp3::steps::Demixer, dp3::steps::DDECal, dp3::steps::Counter, dp3::steps::Clipper, dp3::steps::BDAResultStep, dp3::steps::BdaGroupPredict, dp3::steps::BdaExpander, dp3::steps::BdaDdeCal, dp3::steps::BdaAverager, dp3::steps::Averager, dp3::steps::ApplyCal, dp3::steps::ApplyBeam, dp3::steps::AOFlaggerStep, dp3::steps::AntennaFlagger, dp3::pythondp3::PyStep, dp3::steps::Transfer, dp3::steps::FlagTransfer, dp3::steps::DynSpec, and dp3::steps::Combine.

GetWritableInfoOut()

base::DPInfo& dp3::steps::Step::GetWritableInfoOut ( )

inlineprotected

Returns

Non-const reference to output info.

outputs()

virtual MsType dp3::steps::Step::outputs ( ) const

inlinevirtual

Return which datatype this step outputs.

Reimplemented in dp3::steps::UVWFlagger, dp3::steps::ScaleData, dp3::steps::Predict, dp3::steps::MSBDAReader, dp3::steps::BdaGroupPredict, dp3::steps::BdaDdeCal, and dp3::steps::BdaAverager.

process() [1/2]

virtual bool dp3::steps::Step::process ( std::unique_ptr< base::BdaBuffer >  )

inlinevirtual

Process the BDA data. When processed, it invokes the process function of the next step. It should return False at the end.

Reimplemented in dp3::steps::UVWFlagger, dp3::steps::ScaleData, dp3::steps::Predict, dp3::steps::NullStep, dp3::steps::MSBDAWriter, dp3::steps::DummyStep, dp3::steps::BdaGroupPredict, dp3::steps::BdaExpander, dp3::steps::BdaDdeCal, dp3::steps::Combine, dp3::steps::MockStep, dp3::steps::MsColumnReader, dp3::steps::MSBDAReader, and dp3::steps::BDAResultStep.

process() [2/2]

virtual bool dp3::steps::Step::process ( std::unique_ptr< base::DPBuffer >  buffer )

inlinevirtual

Process the data. When processed, the step should invoke the process function of the next step with the same buffer as argument.

Returns

False at the end of the input. True if there is more input.

Reimplemented in dp3::steps::SVPInput, dp3::steps::SagecalPredict, dp3::steps::UVWFlagger, dp3::steps::ScaleData, dp3::steps::Predict, dp3::steps::OnePredict, dp3::steps::NullStep, dp3::steps::MadFlagger, dp3::steps::IDGImager, dp3::steps::H5ParmPredict, dp3::steps::GainCal, dp3::steps::DummyStep, dp3::steps::Demixer, dp3::steps::Clipper, dp3::pythondp3::PyStep, dp3::steps::Combine, dp3::steps::Interpolate, dp3::steps::WSCleanWriter, dp3::steps::WGridderPredict, dp3::steps::Upsample, dp3::steps::MockStep, dp3::steps::StationAdder, dp3::steps::Split, dp3::steps::SetBeam, dp3::steps::ResultStep, dp3::steps::PreFlagger, dp3::steps::PhaseShift, dp3::steps::OneApplyCal, dp3::steps::NullStokes, dp3::steps::MultiResultStep, dp3::steps::MultiMsReader, dp3::steps::MSWriter, dp3::steps::MSUpdater, dp3::steps::MsReader, dp3::steps::MsColumnReader, dp3::steps::MSBDAReader, dp3::steps::IDGPredict, dp3::steps::Filter, dp3::steps::DDECal, dp3::steps::Counter, dp3::steps::BdaAverager, dp3::steps::Averager, dp3::steps::ApplyCal, dp3::steps::ApplyBeam, dp3::steps::AOFlaggerStep, dp3::steps::AntennaFlagger, dp3::steps::Transfer, dp3::steps::FlagTransfer, and dp3::steps::DynSpec.

setInfo()

void dp3::steps::Step::setInfo

(

const base::DPInfo &

)

Set the info of this step and its next step. It calls the virtual function updateInfo to do the real work.

setNextStep()

virtual void dp3::steps::Step::setNextStep ( Step::ShPtr  nextStep )

inlinevirtual

Set the next step.

Reimplemented in dp3::steps::ApplyCal.

setPrevStep()

void dp3::steps::Step::setPrevStep ( Step *  prevStep )

inline

Set the previous step.

SetThreadingIsInitialized()

static void dp3::steps::Step::SetThreadingIsInitialized ( )

inlinestatic

Prevents that the first Step constructor will initialize the thread pool with the system's number of cpus. This mechanism makes sure that individual steps that would e.g. be created through the Python interface use an appropriate number of threads, while simultaneously making it possible to override the number of threads in a parset, as is done at the start of Dp3.

show()

virtual void dp3::steps::Step::show ( std::ostream &  ) const

pure virtual

Show the step parameters.

Implemented in dp3::steps::AntennaFlagger, dp3::steps::WSCleanWriter, dp3::pythondp3::PyStep, dp3::steps::WGridderPredict, dp3::steps::UVWFlagger, dp3::steps::Upsample, dp3::steps::test::ThrowStep, dp3::steps::MockInput, dp3::steps::SVPInput, dp3::steps::StationAdder, dp3::steps::Split, dp3::steps::SetBeam, dp3::steps::ScaleData, dp3::steps::SagecalPredict, dp3::steps::ResultStep, dp3::steps::PreFlagger, dp3::steps::Predict, dp3::steps::PhaseShift, dp3::steps::OnePredict, dp3::steps::OneApplyCal, dp3::steps::NullStokes, dp3::steps::NullStep, dp3::steps::MultiResultStep, dp3::steps::MultiMsReader, dp3::steps::MSWriter, dp3::steps::MSUpdater, dp3::steps::MsReader, dp3::steps::MsColumnReader, dp3::steps::MSBDAWriter, dp3::steps::MSBDAReader, dp3::steps::MadFlagger, dp3::steps::Interpolate, dp3::steps::IDGPredict, dp3::steps::IDGImager, dp3::steps::H5ParmPredict, dp3::steps::GainCal, dp3::steps::Filter, dp3::steps::DummyStep, dp3::steps::Demixer, dp3::steps::DDECal, dp3::steps::Counter, dp3::steps::Clipper, dp3::steps::BDAResultStep, dp3::steps::BdaGroupPredict, dp3::steps::BdaExpander, dp3::steps::BdaDdeCal, dp3::steps::BdaAverager, dp3::steps::Averager, dp3::steps::ApplyCal, dp3::steps::ApplyBeam, dp3::steps::AOFlaggerStep, dp3::steps::Transfer, dp3::steps::FlagTransfer, dp3::steps::DynSpec, and dp3::steps::Combine.

showCounts()

virtual void dp3::steps::Step::showCounts ( std::ostream &  ) const

virtual

Show the flag counts if needed. The default implementation does nothing.

Reimplemented in dp3::steps::UVWFlagger, dp3::steps::PreFlagger, dp3::steps::MultiMsReader, dp3::steps::MsReader, dp3::steps::MadFlagger, dp3::steps::Demixer, dp3::steps::Counter, and dp3::steps::AOFlaggerStep.

showTimings()

virtual void dp3::steps::Step::showTimings ( std::ostream &  , double  duration  ) const

virtual

Show the timings. The default implementation does nothing.

Reimplemented in dp3::steps::AntennaFlagger, dp3::steps::WSCleanWriter, dp3::steps::SagecalPredict, dp3::steps::MsColumnReader, dp3::steps::WGridderPredict, dp3::steps::UVWFlagger, dp3::steps::StationAdder, dp3::steps::Split, dp3::steps::ScaleData, dp3::steps::PreFlagger, dp3::steps::PhaseShift, dp3::steps::OnePredict, dp3::steps::OneApplyCal, dp3::steps::NullStokes, dp3::steps::MultiMsReader, dp3::steps::MSWriter, dp3::steps::MSUpdater, dp3::steps::MsReader, dp3::steps::MSBDAReader, dp3::steps::MadFlagger, dp3::steps::Interpolate, dp3::steps::IDGPredict, dp3::steps::IDGImager, dp3::steps::H5ParmPredict, dp3::steps::GainCal, dp3::steps::Filter, dp3::steps::DummyStep, dp3::steps::Demixer, dp3::steps::DDECal, dp3::steps::Clipper, dp3::steps::BdaGroupPredict, dp3::steps::BdaExpander, dp3::steps::BdaDdeCal, dp3::steps::Averager, dp3::steps::ApplyCal, dp3::steps::ApplyBeam, dp3::steps::AOFlaggerStep, dp3::steps::Transfer, dp3::steps::FlagTransfer, dp3::steps::DynSpec, and dp3::steps::Combine.

updateInfo()

virtual void dp3::steps::Step::updateInfo ( const base::DPInfo &  )

virtual

Update the general info (called by setInfo). The default implementation copies the info.

Reimplemented in dp3::steps::WSCleanWriter, dp3::steps::WGridderPredict, dp3::steps::SetBeam, dp3::steps::IDGPredict, dp3::steps::AntennaFlagger, dp3::steps::UVWFlagger, dp3::steps::Upsample, dp3::steps::test::ThrowStep, dp3::steps::SVPInput, dp3::steps::StationAdder, dp3::steps::Split, dp3::steps::ScaleData, dp3::steps::SagecalPredict, dp3::steps::PreFlagger, dp3::steps::Predict, dp3::steps::PhaseShift, dp3::steps::OnePredict, dp3::steps::OneApplyCal, dp3::steps::NullStokes, dp3::steps::MultiMsReader, dp3::steps::MSWriter, dp3::steps::MSUpdater, dp3::steps::MsReader, dp3::steps::MsColumnReader, dp3::steps::MSBDAWriter, dp3::steps::MSBDAReader, dp3::steps::MadFlagger, dp3::steps::IDGImager, dp3::steps::H5ParmPredict, dp3::steps::GainCal, dp3::steps::Filter, dp3::steps::DummyStep, dp3::steps::Demixer, dp3::steps::DDECal, dp3::steps::Counter, dp3::steps::Clipper, dp3::steps::BdaGroupPredict, dp3::steps::BdaExpander, dp3::steps::BdaDdeCal, dp3::steps::BdaAverager, dp3::steps::Averager, dp3::steps::ApplyBeam, dp3::steps::AOFlaggerStep, dp3::pythondp3::PyStep, dp3::steps::Transfer, dp3::steps::FlagTransfer, dp3::steps::DynSpec, and dp3::steps::Combine.

Member Data Documentation

kDataField

constexpr dp3::common::Fields dp3::steps::Step::kDataField

staticconstexpr

Initial value:

{
      dp3::common::Fields::Single::kData}

kFlagsField

constexpr dp3::common::Fields dp3::steps::Step::kFlagsField

staticconstexpr

Initial value:

{
      dp3::common::Fields::Single::kFlags}

kUvwField

constexpr dp3::common::Fields dp3::steps::Step::kUvwField

staticconstexpr

Initial value:

{
      dp3::common::Fields::Single::kUvw}

kWeightsField

constexpr dp3::common::Fields dp3::steps::Step::kWeightsField

staticconstexpr

Initial value:

{
      dp3::common::Fields::Single::kWeights}

---

The documentation for this class was generated from the following file:

• Step.h