DDECal

Calibrate direction dependent gains.

inputs

type

type:

string

doc:

Case-insensitive step type; must be ‘ddecal’ .

antennaconstraint

type:

list

default:

[]

doc:

A list of lists specifying groups of antennas that are to be constrained to have the same solution. Example: “[ [CS002HBA0,CS002HBA1],[CS003HBA0,CS003HBA1] ]” will keep the solutions of CS002HBA0 and 1 the same, and the same for CS003 .

applycal.*

doc:

ApplyCal sub-step, same as in Predict step. One can pass an h5parm with as many directions as set in “directions” and each direction model is corrupted accordingly .

approxchunksize

type:

int

default:

0

doc:

Size of fitted chunksize during approximation stage in nr of channels. With approxchunksize=1 the constraint is disabled during the approx stage (so channels are solved for independently). Once converged, the solutions are constrained and more iterations are performed until that has converged too. The default is approxchunksize=0, which calculates the chunksize from the bandwidth (resulting in 10 chunks per octave of bandwidth) .

approximatetec

type:

bool

default:

False

doc:

Uses an approximation stage in which the phases are constrained with the piece-wise fitter, to solve local minima problems. Only effective when mode=tec or mode=tecandphase .

approxtolerance

type:

double

default:

tolerance*10

doc:

Tolerance at which the approximating first stage is considered to be converged and the second full-constraining stage is started. The second stage convergences when the tolerance set by the ‘tolerance’ keyword is reached. Setting approxtolerance to lower values will cause more approximating iterations. Since tolerance is by default 1e-5, approxtolerance is by default 1e-4 .

coreconstraint

type:

double

default:

0

doc:

Distance in meters. When unequal to 0, all stations within the given distance from the reference station (0) will be constraint to have the same solution .

detectstalling

type:

bool

doc:

Stop iterating when no improvement is measured anymore (after a minimum of 30 iterations) .

default:

True

directions

type:

list

default:

[]

doc:

List of directions to calibrate on. Every element of this list should b a list of facets. Default: every facet is a direction .

flagdivergedonly

type:

bool

default:

False

doc:

Flag only the unconverged solutions for which divergence was detected. At the moment, this option is effective only for rotation+diagonal solves, where divergence is detected when the amplitudes of any station are found to be more than a factor of 5 from the mean amplitude over all stations. If divergence for any one station is detected, all stations are flagged for that solution interval. Only effective when flagunconverged=true and mode=rotation+diagonal .

flagunconverged

type:

bool

default:

False

doc:

Flag unconverged solutions (i.e., those from solves that did not converge within maxiter iterations) .

grouppredict

type:

bool

default:

False

doc:

Alternative predict step for BDA data, which groups the input data into similarly averaged baselines and uses a regular Predict step for each group. Due to the overhead of having different groups, the normal predict step processes BDA data faster. This step is therefore mainly used for benchmarking .

h5parm

type:

string

doc:

Filename of output H5Parm (to be read by e.g. losoto). If empty, defaults to instrument.h5 within the measurement set .

idg.buffersize

type:

int

default:

Based on memory

doc:

Set the amount of timesteps that are to be used for each IDG buffer .

idg.images

type:

list

doc:

Filename of .fits model images, one per frequency term. The terms are defined as for a polynomial source spectra (not logarithmic), e.g. see this WSClean page. The frequency in the metadata of the fits files is used as nu₀ in the polynomial evaluation .

default:

[]

idg.regions

type:

string

default:

""

doc:

DS9 regions file describing the facets for IDG prediction .

keep_host_buffers

type:

bool

default:

False

doc:

Setting this to true will cause the host buffers to be kept during the execution if used in combination with the GPU solver. This can cause an higher consumption of memory but saves execution time.

keepmodel

type:

bool

default:

False

doc:

If true, pass the uncorrected model data to the next step. DDECal stores the model data using its step name plus the (first) direction name. For example, if “steps=[ddecal]” and “ddecal.directions=[[center,dec_off],[ra_off]]”, the model data names are “ddecal.center” and “ddecal.ra_off”. If false, delete all model data, including the model data from “reusemodel” .

llssolver

type:

string

default:

qr

doc:

Select which linear least-squares solver to use for the direction-solving algorithm. When solveralgorithm is not directionsolve, this parameter has no effect. Supported least-squares solvers are qr, svd and normalequations .

maxapproxiter

type:

int

default:

maxiter/2

doc:

Maximum number of iterations during approximating stage .

maxiter

type:

int

default:

50

doc:

Maximum number of iterations .

minvisratio

type:

double

default:

0

doc:

Minimum number of visibilities within a solution interval, e.g. 0.6 for at least 60% unflagged vis. Intervals with fewer vis will be flagged .

mode

type:

string

default:

diagonal

doc:

Solve type. Options are scalar, scalarphase, scalaramplitude, diagonal, diagonalphase, diagonalamplitude, fulljones, tec, tecandphase, rotation and rotation+diagonal .

modeldatacolumns

type:

list

default:

[]

doc:

Use model data from the measurement set. All directions from this parameter will be pointing to the measurement set metadata .

modelnextsteps

type:

list

default:

[]

doc:

Defines the steps that will be applied to the model in every direction before using the output for calibration.

modelnextsteps

type:

string

doc:

Allows steps to be defined for a specific direction. These directions, (given in the <direction> placeholder) can be specified by their model column name, the name of a DS9 region, or the first patch name in the direction if using a sky model. For example: modelnextsteps.MODEL_DATA_2=[applyextrabeam], applyextrabeam.type=applybeam .

nchan

type:

int

doc:

Number of channels in each channel block, for which the solution is assumed to be constant. The default is 1, meaning one solution per channel (or in the case of constraints, fitting the constraint over all channels individually). 0 means one solution for the whole channel range. If the total number of channels is not divisable by nchan, some channelblocks will become slightly larger. When using BDA, the value of nchan should be bigger than or equal to the maximum number of averaged frequency channels .

default:

1

onlypredict

type:

bool

default:

False

doc:

Instead of solving, output the predicted visibilities instead. This is useful for testing, although when doing faceted prediction with IDG, it might be fast for certain cases .

phasereference

type:

bool

default:

True

doc:

Use phase referencing. This setting is only effective when mode=tec or mode=tecandphase .

propagateconvergedonly

type:

bool

default:

False

doc:

Propagate solutions of the previous time slot only if the solve converged. Only effective when propagatesolutions=true .

propagatesolutions

type:

bool

default:

False

doc:

Initialize solver with the solutions of the previous time slot .

reusemodel

type:

list

default:

[]

doc:

Reuse model data from previous (ddecal) step(s). The list should contain the model data names, as set by a previous step, e.g., “[ddecal1.MODEL_DATA, ddecal1.center]” .

rotationreference

type:

bool

default:

False

doc:

Use rotation referencing with the first antenna as the reference antenna. When enabled, the first antenna gets a rotation angle of 0. The rotation angle of the other antennas will be relative to the first antenna. If the first antenna has no valid data, the second antenna will be the reference antenna, and so on. This setting is only effective when mode=rotation+diagonal .

savefacets

type:

bool

default:

False

doc:

Write out each facet as a fits file (named facet<N>.fits). Only useful using idg .

smoothnessconstraint

type:

double

default:

0.0

doc:

Kernel size in Hz. When unequal to 0, will constrain the solutions to be smooth over frequency by convolving the solutions with a kernel of the given size (bandwidth). The default kernel is a Gaussian kernel, and the kernel size parameter is the 3 sigma point from the centre where the kernel is cut off. For example, with a kernel size of 2 MHz and with 1 MHz channels, the kernel will cover 2 channels on both side from the centre, and therefore 5 channels in total .

smoothnessrefdistance

type:

double

default:

0.0

doc:

An optional reference distance (in meters) for the smoothness constraint. When unequal to 0, the size of the smoothing kernel will vary, such that antennas (stations for LOFAR) closer to antenna 0 will be smoothed stronger, i.e. with a larger kernel size. .

smoothnessreffrequency

type:

double

default:

0.0

doc:

An optional reference frequency (in Hz) for the smoothness constraint. When unequal to 0, the size of the smoothing kernel will vary over frequency by a factor of “frequency / smoothnessreffrequency”, i.e., the kernel will be smaller for lower frequencies. This compensates for the fact that the ionosphere causes larger fluctuations at lower frequencies .

solint

type:

int

doc:

Solution interval in timesteps. If a value of 0 is specified, the solver will use a single solution interval that covers the entire observation. When using BDA, the solint should be bigger than or equal to the maximum averaged time interval .

default:

1

solutions_per_direction

type:

list

default:

[1, 1, ..., 1]

doc:

List of number of subsolutions per solution interval per direction. Defaults to a single solution per solution interval per direction. Entries in provided list should be integer divisors of solint. If not all directions are specified, missing directions are solved with one solution per solution interval. Direction-dependent intervals are only supported when solveralgorithm is set to directioniterative .

solveralgorithm

type:

string

doc:

Select the algorithm used for the solving. Options are directionsolve, directioniterative, lbfgs, and hybrid. For 50 directions, the direction-iterating algorithm can be an order of magnitude faster than the direction-solving algorithm. The downside might be that it is less stable and/or less accurate, which is still to be tested. All algorithms support all solver modes (scalar, diagonal, full Jones, etc). The use of direction-dependent solution intervals (see solutions_per_direction) is only supported for the directioniterative method. The hybrid algorithm is the best of both. It runs 1/6th of the maximum iterations with the direction-solving algorithm, and then switches to the direction-iterating .

default:

directionsolve

solverlbfgs.dof

type:

double

default:

200.0

doc:

The degrees of freedom of the robust noise model .

solverlbfgs.history

type:

int

doc:

LBFGS solver history size. This is the memory (as a multiple of the number of parameters) used to store information pertaining to previous iterations .

default:

10

solverlbfgs.iter

type:

int

default:

4

doc:

Iterations per each minibatch of data. This is the inner iterations and maxiter is the outer iterations. Therefore, the total number iterations is the product of these two values .

solverlbfgs.minibatches

type:

int

default:

1

doc:

Divide the data into minibatches equal to this number at each iteration. If this value is greater than one, less data is used at each iteration (without losing the final solution accuracy) making the iterations faster .

sourcedb

type:

string

doc:

Sourcedb (created with makesourcedb) with the sky model to calibrate on . When the path ends with .skymodel or .txt DP3 expects a skymodel file as used by makesourcedb. This makes it possible to directly use a skymodel file without using makesourcedb to convert the file.

statfilename

type:

string

doc:

File to write the step-sizes to. Form of the file is: “<iterationnr> <normalized-stepsize> <unnormalized-stepsize>”, and all solution intervals are concatenated. File is not written when this parameter is empty .

stepsigma

type:

double

default:

0.1

doc:

Threshold for stalling detection, stop iterating when the running mean of the step sizes is less than stepsigma times their running standard deviation, i.e., the step size is just noise .

stepsize

type:

double

default:

0.2

doc:

stepsize between iterations .

storebuffer

type:

bool

doc:

Setting this to true will store the solution of DDECal into the buffer, allowing the usage of this solution in a later step. For example, a pipeline with  DDECal -> OneApplyCal  would be able to apply solutions to the data without requiring an intermediate format to be stored to disk. Note that it currently only works for single-direction solutions.

default:

False

subtract

type:

bool

default:

False

doc:

Subtracts the corrected model from the data. NOTE This may not work when you apply a uv-cut .

tolerance

type:

double

default:

1e-5

doc:

Controls the accuracy to be reached: when the normalized solutions move less than this value, the solutions are considered to be converged and the algorithm finishes. Lower values will cause more iterations to be performed .

usebeammodel

type:

bool

default:

False

doc:

use the beam model. All beam-related options of the Predict step are also valid .

usegpu

type:

bool

default:

False

doc:

Use GPU solver. This is an experimental feature only available for the iterative diagonal solver and requires DP3 to be built with BUILD_WITH_CUDA=1 .

uvlambdamin

type:

double

default:

0

doc:

Ignore baselines / channels with UV < uvlambdamin wavelengths. Note: also all other variants of uv flagging described in UVWFlagger (uvmmin, uvmrange, uvlambdarange, etc) are supported (New in 3.1) .