User Parameters - Continuum imaging¶
Summary¶
The parameters listed here allow the user to configure the parset and the slurm job for continuum imaging of the science field. The defaults have been set based on experience with science commissioning and ASKAP Early Science, using the 12-antenna array. If you are processing different data sets, adjusting these defaults may be necessary.
Some of the parameters can be set to blank, to allow cimager to choose appropriate values based on its “advise” capability (which involves examining the dataset and setting appropriate values for some parameters.
For most observations to date, there is a single phase centre that
applies for all beams. Thus, if advise is used to determine the
image centre, the same direction will be used for each beam. This is
the way the BETA processing was done, and in this case the image size
must be chosen to be big enough to encompass all beams. This behaviour
is applied when IMAGE_AT_BEAM_CENTRES=false
.
The preferred behaviour, however, is to set a different image centre
per beam, as this allows a smaller image to be made when imaging. To
do this, the footprint.py tool is used to determine the locations of
the centres of each beam. The footprint specification is determined
preferentially from the scheduling block parset, or (if not available
there) from the parameters described at
User Parameters - Mosaicking. To use this mode, set
IMAGE_AT_BEAM_CENTRES=true
.
While the default application used for the imaging is cimager, it is possible to use the newer imager (imager). Much of the functionality will be identical for continuum data (imager was developed initially as a better spectral-line imaging tool).
Self-calibration¶
The default approach for continuum imaging is to do it in conjunction with self-calibration. The algorithm here is as follows:
Image the data with cimager or imager
Run source-finding with Selavy with a relatively large threshold (unless we are using option 3c. below)
Use the results to calibrate the antenna-based gains by either:
Create a component parset from the resulting component catalogue and use this parset in ccalibrator
Create a model image from the component catalogue, and use in ccalibrator
Use the clean model image from the most recent imaging as the model for ccalibrator (no source-finding will be done)
Re-run imaging, applying the latest gains table
Repeat steps 2-5 for a given number of loops
Each loop gets its own directory, where the intermediate images, source-finding results, and gains calibration table are stored. At the end, the final gains calibration table is kept in the main output directory (as this can then be used by the spectral-line imaging pipeline).
There are two options for how this is run. The first
(MULTI_JOB_SELFCAL=true
) launches a separate slurm job for each
imaging task, and each “calibration” task - the calibration job
incorporates the selavy, cmodel and ccalibrator tasks. The latter uses
a single node only. If MULTI_JOB_SELFCAL=false
, these are all
incorporated into one slurm job, including all loops.
Some parameters are allowed to vary with the loop number of the self-calibration. This way, you can decrease, say, the detection threshold, or increase the number of major cycles, as the calibration steadily improves. These parameters are for:
Deconvolution:
CLEAN_ALGORITHM
,CLEAN_GAIN
,CLEAN_PSFWIDTH
,CLEAN_THRESHOLD_MAJORCYCLE
,CLEAN_NUM_MAJORCYCLES
,CLEAN_MINORCYCLE_NITER
,CLEAN_THRESHOLD_MINORCYCLE
andCLEAN_SCALES
Preconditioning:
PRECONDITIONER_LIST
,PRECONDITIONER_GAUSS_TAPER
,PRECONDITIONER_GAUSS_TAPER_IS_PSF
,PRECONDITIONER_GAUSS_TAPER_TOL
,PRECONDITIONER_WIENER_ROBUSTNESS
, andPRECONDITIONER_WIENER_TAPER
Self-calibration:
SELFCAL_SELAVY_THRESHOLD
,SELFCAL_INTERVAL
andSELFCAL_NORMALISE_GAINS
Data selection in imaging:
CIMAGER_MINUV
andCIMAGER_MAXUV
Data selection in calibration:
CCALIBRATOR_MINUV
andCCALIBRATOR_MAXUV
To use this mode, the values for these parameters should be given as
an array in the form SELFCAL_INTERVAL="[1800,1800,900,300]"
, or,
when the parameter would take an array of values, those arrays should
be separated by semi-colons (with a space on either side):
CLEAN_SCALES="[0] ; [0,20] ; [0,20,120,240] ; [0,20,120,240,480]"
.
The size of these arrays should be one more than
SELFCAL_NUM_LOOPS
. This is because loop 0 is just the imaging, and
it is followed by SELFCAL_NUM_LOOPS
loops of
source-find–model–calibrate–image. Only the parameters related to
the imaging (the deconvolution & preconditioning parameters in the list above) have
the first element of their array used. If a single value is given for
these parameters, it is used for every loop.
Instead of preconditioning, it is possible to use traditional
weighting through the the use of TRADITIONAL_WEIGHT_PARAM_LIST
and
TRADITIONAL_WEIGHT_ROBUSTNESS
, which can be loop dependent as
well.
Once the gains solution has been determined, it can be applied directly to the continuum measurement set, creating a copy in the process. This is necessary for continuum cube processing, and for archiving purposes. This work is done as a separate slurm job, that starts upon completion of the self-calibration job.
There is a prototype option available (using
SELFCAL_MASK_CLEAN_MODEL
) that allows the clean model that is used
for calibration to be modified. Initially, this process just applies a
single threshold to the image, masking out pixels below that
threshold, but the aim is to add further refinements to this
process. This only alters the image used by the calibration task.
External calibration¶
A prototype option has been provided to allow the selfcal process
to start with a phase-only calibration against a user-provided source
catalogue (using the EXTERNAL_CATALOGUE
parameter).
This catalogue is used to create a model image, which is
then used by ccalibrator to do phase-only calibration of the
data. This is done prior to imaging, so that the first image has
phases calibrated to the positions of sources in the catalogue. This
can help improve the quality of the first image used in the
selfcal. Self-calibration then proceeds as normal, with subsequent
calibration steps using the model derived from that initial image.
Two special values for EXTERNAL_CATALOGUE
are recognised:
racs_mid
or racs_low
, which allow the querying of the
respective RACS whole-sky catalogues through the sky model service
interface in cmodel
. This mode also provides for the application
of primary-beam attenuation via the same interface used in linmos.
Leakage correction¶
The pipeline now has a feature to optionally derive and correct for
the effect of on-axis leakages of unpolarised ssources to the the other
Stokes parameters. The leakages are derived from the bandpass-corrected,
spectrally-averaged and flagged PKS1934-638 data. Users need to specify
DO_LEAKAGE_CAL_CONT=true
to derive the antenna leakages, and specify
DO_APPLY_LEAKAGE=true
to apply the leakage corrections to uvdata.
The leakage correction is applied to gain-corrected data obtained after
the self-calibration.
Further imaging : polarisation and cubes¶
Following this application of the gains and/or leakage calibration solution, one can optionally image the continuum dataset as a cube, preserving the frequency sampling. This task also allows the specification of multiple polarisations, with a cube created for each polarisation given.
A prototype option is available (using
DO_MFS_STARTING_MODEL_CONTCUBE
) to use the model image from the
MFS (continuum) imaging as the starting point for the clean model in
the continuum cube imaging.
Following continuum-cube imaging, the cube statistics are calculated, using a distributed task within the same slurm job as the spectral imaging. This produces a file listing a series of statistics for each channel, as well as a plot of the statistics. See Validation and Diagnostics for examples. These statistics are then used to identify problematic channels (for example, due to divergence in the imaging caused by RFI) that are then masked. Blank channels (those not imaged, e.g. due to barycentric correction or missing data) are also masked. The statistics file is then regenerated.
A note on the imagers and the output formats. The default approach is
to now use imager (imager) for both continuum and
spectral imaging. The use of simager for spectral imaging (in this
case, the continuum cubes) is to be discouraged, as imager provides
much better functionality for integration into the pipeline. The use
of cimager for the MFS imaging is possible, although it provides
less flexibility in how the job can be distributed across nodes. The
choice of imaging task is governed by DO_ALT_IMAGER_CONT
or
DO_ALT_IMAGER_CONTCUBE
, with the main switch DO_ALT_IMAGER
taking precedence.
The default output format is FITS images, although CASA files can be
written instead by setting IMAGETYPE_CONT
or
IMAGETYPE_CONTCUBE
to casa
(rather than fits
).
This version allows continuum imaging of various polarisation products.
The self-calibration stage is used for Stokes-I imaging. The calibrated
uv-data is then used to make images of other polarisations. Users need
to specify the list of polarisation using CONTIMG_POLARISATIONS
in
their pipeline configuration file. In a later version, we aim to separate
the self-calibration from the imaging, so that all pols including Stokes-I
are imaged along with the other polarisations.
Basic imaging parameters¶
This table summarises the fundamental parameters that govern the continuum imaging, and that are used in most of the different types of imaging jobs. This also has specific parameters related to the use of imager.
Variable |
Default |
Parset equivalent |
Description |
---|---|---|---|
|
true |
none |
Whether to image the science MS |
|
|
none |
Time request for imaging the continuum (both types - with and without self-calibration) |
|
fits |
Image format to use - can be either ‘casa’ or ‘fits’. |
|
|
fits |
imagetype (imager) |
Image format to use - can be either ‘casa’ or ‘fits’,
although ‘fits’ can only be given in conjunction with
|
|
false |
none |
Whether to break the selfcal up into separate slurm jobs for
each imaging and calibration task ( |
|
|
none |
Time request for the calibration jobs when running with
|
Basic variables |
|||
|
true |
none |
Whether to have each beam’s image centred at the centre of the beam (IMAGE_AT_BEAM_CENTRES=true), or whether to use a single image centre for all beams. |
|
|
none |
The number of cores in total to use for the continuum
imaging. If left blank ( |
|
|
Channels (Data Selection) |
If |
|
6 (galaxy), 36 (setonix/petrichor) |
Not for parset |
Number of cores to use on each node in the continuum imaging. |
|
true |
Not for parset |
Whether the master process for the continuum imaging should be
put on a node of its own (if |
|
DATA |
datacolumn (cimager) |
The column in the measurement set from which to read the visibility data. The default, ‘DATA’, is appropriate for datasets processed within askapsoft, but if you are trying to image data processed, for instance, in CASA, then changing this to CORRECTED_DATA may be what you want. |
|
i.%t.SB%s.cont |
Helps form Images.Names (cimager) |
The base name for images: if |
|
none |
Images.<imagename>.direction (cimager) |
The direction parameter for the images, i.e. the central position. Can be left out, in which case Cimager will get it from either the beam location (for IMAGE_AT_BEAM_CENTRES=true) or from the measurement set using the “advise” functionality (for IMAGE_AT_BEAM_CENTRES=false). |
|
6144 |
Images.shape (cimager) |
The number of pixels on the side of the images to be created.
If negative, zero, or absent (i.e. |
|
2 |
Images.cellsize (cimager) |
Size of the pixels in arcsec. If negative, zero or absent, this will be set automatically by the Cimager “advise” function, based on examination of the MS. The default is chosen together with the default number of pixels to cover a typical ASKAP beam with the sidelobes being imaged. |
|
2 |
Images.image.${imageBase}.nterms (cimager) linmos.nterms (linmos (Linear Mosaic Applicator)) |
Number of Taylor terms to create in MFS imaging. If more than 1, MFS weighting will be used (equivalent to setting Cimager.visweights=MFS in the cimager parset). |
|
no default |
visweights.MFS.reffreq (cimager) |
Frequency at which continuum image is made [Hz]. This is the reference frequency for the multi-frequency synthesis, which should usually be the middle of the band. If negative, zero, or absent (the default), this will be set automatically to the average of the frequencies being processed. |
|
fit |
Restoring beam to use: ‘fit’ will fit the PSF to determine
the appropriate beam, else give a size (such as
|
|
|
0.5 |
restore.beam.cutoff (simager) |
Cutoff value used in determining the support for the fitting (ie. the rectangular area given to the fitting routine). Value is a fraction of the peak. |
|
0 |
MinUV (Data Selection) |
The minimum UV distance considered in the imaging - used to
exclude the short baselines. Can be given as an array with
different values for each self-cal loop
(e.g. |
|
0 |
MaxUV (Data Selection) |
The maximum UV distance considered in the imaging. Only used
if greater than zero. Can be given as an array with different
values for each self-cal loop (e.g. |
Gridding parameters |
|||
|
MPIWProject |
gridder (Gridders) |
Specify griddder to use: WProject or MPIWProject MPIWProject can save memory and time when using multiple ranks |
|
true |
Images.nyquistgridding (imager) |
Whether to turn on Nyquist gridding. |
|
false |
snapshotimaging (Gridders) |
Whether to use snapshot imaging when gridding. |
|
2600 |
snapshotimaging.wtolerance (Gridders) |
The wtolerance parameter controlling how frequently to snapshot. |
|
true |
snapshotimaging.longtrack (Gridders) |
The longtrack parameter controlling how the best-fit W plane is determined when using snapshots. |
|
0.01 |
snapshotimaging.clipping (Gridders) |
If greater than zero, this fraction of the full image width is set to zero. Useful when imaging at high declination as the edges can generate artefacts. |
|
2600
( |
MPIWProject.wmax (Gridders) |
The wmax parameter for the gridder. The default for this
depends on whether snapshot imaging is invoked or not
( |
|
false |
MPIWProject.wmaxclip (Gridders) |
The wmaxclip parameter, used for all gridders. By default, a hard cut for w planes on [-wmax,wmax] is made and values outside this range will trigger an error. When true, w values outside this range are ignored. |
|
99
( |
MPIWProject.nwplanes (Gridders) |
The nwplanes parameter for the gridder. The default for this
depends on whether snapshot imaging is invoked or not
( |
|
8 |
MPIWProject.oversample (Gridders) |
The oversampling factor for the gridder. |
|
512
( |
MPIWProject.maxsupport (Gridders) |
The maxsupport parameter for the gridder. The default for this
depends on whether snapshot imaging is invoked or not
( |
|
true |
MPIWProject.sharecf (Gridders) |
Whether to use a (static) cache for the convolution functions in the WProject gridder. |
|
16 |
MPIWProject.cfrank (Gridders) |
Number of ranks per node to use to speed up convolution function calculation. Useful range: ~4 to #ranks/node |
|
false |
Cimager.sphfuncforoffset (cimager) |
If true, the default spheroidal function gridder is used to grid offset fields regardless of the gridder selected for model degridding and residual gridding. |
Preconditioning / weighting parameters |
|||
|
|
preconditioner.Names (Solvers) |
List of preconditioners to apply. Can be varied for each
selfcal cycle (e.g. |
|
|
preconditioner.GaussianTaper (Solvers) |
Size of the Gaussian taper - either single value (for circular taper) or 3 values giving an elliptical size. Can be varied for each selfcal cycle. |
|
|
preconditioner.GaussianTaper.isPsfSize (Solvers) |
Decide if rather than applying the specified taper, the taper should be adjusted to ensure that the output image has the specified resolution. NB: This parameter is also passed to coarse cube imaging. Can be varied for each selfcal cycle. |
|
0.005 |
preconditioner.GaussianTaper.tolerance (Solvers) |
Fractional tolerance for the fitted beam size when
|
|
-0.5 |
preconditioner.Wiener.robustness (Solvers) |
Robustness value for the Wiener filter. Can be varied for each selfcal cycle. |
|
|
preconditioner.Wiener.taper (Solvers) |
Size of gaussian taper applied in image domain to Wiener filter. Ignored if blank (ie. “”). Can be varied for each selfcal cycle. |
|
|
List of preconditioners to apply at the restore stage, to produce an additional restored image. |
|
|
|
The additional suffix or extension added to the image name for the additional restored images with alternate preconditioning. |
|
|
|
Size of the Gaussian taper for the restore preconditioning - either single value (for circular taper) or 3 values giving an elliptical size. |
|
|
|
preconditioner.GaussianTaper.isPsfSize (Solvers) |
Decide if rather than applying the specified taper for images restored with alternate preconditioning, the taper should be adjusted to ensure that the output image has the specified resolution. |
|
0.005 |
preconditioner.GaussianTaper.tolerance (Solvers) |
Fractional tolerance for the fitted beam size when
|
|
-2 |
restore.preconditioner.Wiener.robustness (cimager & Solvers) |
Robustness value for the Wiener filter in the restore preconditioning. |
|
|
Size of gaussian taper applied in image domain to Wiener filter in the restore preconditioning. Ignored if blank (ie. “”). |
|
|
|
uvweight (imager) |
List of parameters for application of traditional weights. |
|
-0.5 |
uvweight.robustness (imager) |
The value of robustness. Only used if there is Robust among
the list of parameters given by
|
*New imager parameters |
|||
|
|
none |
If true, the continuum imaging is done by imager
(imager). If false, it is done by cimager
(cimager). If left blank (the default), the
value is given by the overall parameter |
Self-calibration¶
This table summarises the parameters related to stokes-I imaging and the self-calibration procedure. The Stokes I imaging gets its own set of deconvolution (cleaning) parameters (the Stokes I images will be made in the self-cal part, not the “contpol” part of the pipeline). There are parameters related to the source-finding and calibration parts of the self-calibration
Variable |
Default |
Parset equivalent |
Description |
---|---|---|---|
Cleaning parameters |
|||
|
Clean |
Which solver to use. You will mostly want to leave this as ‘Clean’, but there is a ‘Dirty’ solver available. |
|
|
BasisfunctionMFS |
Clean.algorithm (Solvers) |
The name(s) of clean algorithm(s) to use.
To use different algorithms in different selfcal cycles, use:
|
|
|
Clean.niter (Solvers) |
The number of iterations for the minor cycle clean. Can be
varied for each selfcal cycle. (e.g. |
|
0.2 |
Clean.gain (Solvers) |
The loop gain (fraction of peak subtracted per minor cycle).
Can be varied for each selfcal
cycle. (e.g. |
|
256 |
Clean.psfwidth (Solvers) |
The width of the psf patch used in the minor cycle. Can be
varied for each selfcal cycle. (e.g. |
|
|
Clean.scales (Solvers) |
Set of scales (in pixels) to use with the multi-scale clean.
Can be varied for each selfcal cycle (e.g. |
|
|
Threshold for the minor cycle loop. Can be varied for each
selfcal cycle. (e.g. |
|
|
|
The target peak residual. Major cycles stop if this is
reached. A negative number ensures all major cycles requested
are done. Can be given as an array with different values for
each self-cal loop (e.g. |
|
|
|
ncycles (cimager) |
Number of major cycles. Can be given as an array with
different values for each self-cal loop (e.g. |
|
false |
Images.writeAtMajorCycle (cimager) |
If true, the intermediate images will be written (with a .cycle suffix) after the end of each major cycle. |
|
MAXBASE |
Clean.solutiontype (see discussion at Multi-Scale and/or Multi-Frequency deconvolution) |
The type of peak finding algorithm to use in the deconvolution. Choices are MAXCHISQ, MAXTERM0, or MAXBASE. |
|
true |
Clean.detectdivergence (Solvers) |
Whether to detect that the deconvolution is starting to diverge (in which case it is stopped). |
|
true |
Clean.detectmilddivergence (Solvers) |
Whether to use the mild form of divergence detection, that should skip to the next major cycle if residuals increase by 10% or more. |
|
“[1.1,2.0,2.0,0.05]” |
Clean.divergencelevels (Solvers) |
Setting the levels for divergence detection to be used in the continuum imaging. |
|
0 |
Clean.noiseboxsize(Solvers) |
Size of the box used to determine the local noise value for setting thresholds (when a “sigma” threshold is given). A value of 0 means the entire image is used. |
Self-calibration - basics |
|||
|
true |
none |
Whether to self-calibrate the science data when imaging. |
|
|
NONE |
If provided, this catalogue is used to generate a model that is then used to provide phase-only calibration prior to the first round of imaging. Special values of this are “racs_mid” or “racs_low”, to search those whole-sky catalogues through the online Sky Model Service interface in cmodel. |
|
Cmodel |
none |
How to do the self-calibration. There are three options: “Cmodel” means create a model image from the source-finding results; “Components” means use the detected components directly through a parset (created by Selavy); “CleanModel” means use the clean model image from the most recent imaging as the model for ccalibrator. Anything else will default to “Cmodel”. |
|
true |
Cmodel.nearest (cmodel) |
For |
|
SVD |
solver (Calibration solvers) |
Selection of solver - either “SVD” or “LSQR” |
|
0.01 |
solver (Calibration solvers) |
Damping parameter for “LSQR” solver. Ignored for SVD. |
|
1 |
none |
Number of loops of self-calibration. |
|
|
interval (ccalibrator) |
Interval [sec] over which to solve for self-calibration. Can be given as an array with different values for each self-cal loop, as for the default, or a single value that applies to each loop. |
|
calparameters.selfcal.%t.SB%s.%b.tab |
none (directly) |
The table name to hold the final gains solution. Once
the self-cal loops have completed, the cal table in the
final loop is copied to a table of this name in the base
directory. This can then be used for the spectral-line
imaging if need be. If this is blank, both |
|
true |
normalisegains (ccalibrator) |
Whether to normalise the amplitudes of the gains to 1, approximating the phase-only self-calibration approach. Can be given as an array with different values for each self-cal loop (e.g. “[true,true,false]”). |
|
false |
calibrate.interpolatetime (imager, cimager and ccalapply (Calibration Applicator)) |
Whether to interpolate the self-cal gains (amp and pha). If true, interpolated solutions will be used for the interim imaging within the selfcal loops, and in the final stages of calibration application to the continuum and spectral-line measurement sets used for the respective cube imaging. Only linear interpolation is allowed. |
|
|
refantenna (ccalibrator) |
Reference antenna to use in the calibration. Should be antenna number, 0 - nAnt-1, that matches the antenna numbering in the MS. |
|
|
refgains (ccalibrator) |
Reference gains to use in the calibration - something like gain.g11.0.0. |
|
false |
calibrate.scalenoise (cimager) |
Whether the noise estimate will be scaled in accordance with the applied calibrator factor to achieve proper weighting. |
|
0 |
MinUV (Data Selection) |
The minimum UV distance considered in the calibration - used
to exclude the short baselines. Can be given as an array with
different values for each self-cal loop
(e.g. |
|
0 |
MaxUV (Data Selection) |
The maximum UV distance considered in the calibration. Only
used if greater than zero. Can be given as an array with
different values for each self-cal loop
(e.g. |
|
true |
none |
Should we keep the images from the intermediate selfcal loops? |
|
false |
none |
Should we make full-field mosaics for each loop of the self-calibration? This is done for each field separately. |
|
false |
none |
If true, a copy of the clean model image is made to be used for calibration, and it is then masked to remove pixels below a certain threshold - to avoid negative or weak components. |
|
0.001 |
none |
The threshold below which pixels are masked in the modified clean model image. Units are given by the next parameter. |
|
absolute |
none |
The units of the threshold: ‘absolute’ means the native image flux values (typically Jy/pixel); ‘sigmaBased’ means the threshold is in multiples of the image noise. |
Self-calibration - source-finding |
|||
|
8 |
snrCut (Selavy Basics) |
SNR threshold for detection with Selavy in determining
selfcal sources. Can be given as an array with different
values for each self-cal loop (e.g. |
|
6 |
nsubx (Selavy Basics) |
Division of image in x-direction for source-finding in selfcal. |
|
3 |
nsuby (Selavy Basics) |
Division of image in y-direction for source-finding in selfcal. |
|
true |
Selavy.Fitter.numGaussFromGuess (Post-processing of detections) |
Whether to fit the number of Gaussians given by the initial
estimate (true), or to only fit a fixed number (false). The
number is given by |
|
1 |
Selavy.Fitter.maxNumGauss (Post-processing of detections) |
The number of Gaussians to fit to each island when
|
|
full |
Selavy.Fitter.fitTypes (Post-processing of detections) |
The type of fit to be used in the Selavy job. The possible options are ‘full’, ‘psf’, ‘shape’, or ‘height’. |
|
0.95 |
Selavy.Weights.weightsCutoff (Thresholds in Selavy) |
Pixels with weight less than this fraction of the peak weight will not be considered by the source-finding. If the value is negative, or more than one, no consideration of the weight is made. |
|
“” |
spectralTerms.threshold (Post-processing of detections) |
Threshold applied to component peak fluxes in determining
which have a spectral index (and curvature) value reported
in the component catalogue. Not used if left blank. Takes
precedence over
|
|
spectralTerms.thresholdSNR (Post-processing of detections) |
Threshold applied to component peak signal-to-noise values in determining which have a spectral index (and curvature) value reported in the component catalogue. Not used if left blank. |
|
|
10 |
Used to create Cmodel.flux_limit (cmodel) |
The signal-to-noise level used to set the flux limit for
components that are used by Cmodel. The image noise values
reported for all components are averaged, then multiplied by
this value to form the Cmodel flux limit. If left blank
( |
|
10uJy |
Cmodel.flux_limit (cmodel) |
The minimum integrated flux for components to be included in the model used for self-calibration. |
|
false |
none |
Whether to add a fixed RA & Dec offset to the positions of sources in the final self-calibration catalogue (prior to it being used to calibrate the data). This has been implemented to help with commissioning - do not use unless you understand what it is doing! This makes use of the script legacy/fix_position_offsets.py. |
|
0 |
none |
The offset in position in the RA direction, in arcsec. This is taken from the offset_pipeline_params.txt file produced by the continuum validation script, where the sense of the offset is REFERENCE-ASKAP. |
|
0 |
none |
The offset in position in the DEC direction, in arcsec. This is taken from the offset_pipeline_params.txt file produced by the continuum validation script, where the sense of the offset is REFERENCE-ASKAP. |
Applying the gains and leakage calibration¶
This table shows the parameters relevant to the jobs that apply the self-cal gains to the averaged dataset, and then the leakages to the gain-refined measurement sets. This is done before the continuum-polarisation imaging and the continuum-cube imaging.
Variable |
Default |
Parset equivalent |
Description |
---|---|---|---|
Application of gains calibration |
|||
|
true |
none |
Whether to apply the calibration to the averaged (“continuum”) dataset. |
|
|
none |
Time request for applying the calibration |
|
true |
none |
Whether to make a copy of the averaged MS before applying the gains calibration (true), or to just overwrite with the calibrated data (false). |
|
false |
none |
Whether to apply the leakage calibration to the averaged and
self-calibrated (“continuum”) dataset derived using:
|
Additionally, instrumental leakage of Stokes-I into other Stokes parameters in off-axis regions of the beams can be corrected during mosaicking. For more details, see the pipeline documentation on linear mosaicking at User Parameters - Mosaicking and code documentation at linmos (Linear Mosaic Applicator).
Continuum Polarisation imaging¶
Once the gains have been applied and/or leakages have been corrected, MFS imaging
may be done in a selected set of polarisations - use DO_CONTPOL_IMAGING=true
to
turn this mode on.
If “I” is given as one of them, it is not re-imaged, as we have an image from the self-calibration routine. The cleaning parameters are given separately here (as thresholds may need to be different), although gridding and preconditioning are assumed to be the same. Many of these parameters can be given as arrays, with different values for each polarisation. If a single value is given, that applies to all polarisations.
Variable |
Default |
Parset equivalent |
Description |
---|---|---|---|
Imaging parameters |
|||
|
false |
none |
Whether to create continuum polarisation images |
|
|
ImageName.polarisation (cimager) |
Comma separated list of Polarisations to be imaged Stokes-I is ignored here, since it is imaged in the self-cal loop |
|
0 |
MinUV (Data Selection) |
The minimum UV distance considered in the imaging - used to
exclude the short baselines. Can be given as an array with
different values for each pol loop
(e.g. |
|
0 |
MaxUV (Data Selection) |
The maximum UV distance considered in the imaging. Only used
if greater than zero. Can be given as an array with different
values for each pol loop (e.g. |
Cleaning parameters |
|||
|
BasisfunctionMFS |
Clean.algorithm (Solvers) |
The name(s) of clean algorithm(s) to use.
To use different algorithms in different selfcal cycles, use:
|
|
|
Clean.niter (Solvers) |
The number of iterations for the minor cycle clean. Can be
varied for each pol. (e.g. |
|
0.2 |
Clean.gain (Solvers) |
The loop gain (fraction of peak subtracted per minor cycle).
Can be varied for each pol
cycle. (e.g. |
|
256 |
Clean.psfwidth (Solvers) |
The width of the psf patch used in the minor cycle. Can be
varied for each pol (e.g. |
|
|
Clean.scales (Solvers) |
Set of scales (in pixels) to use with the multi-scale clean.
Can be varied for each specified polarisation (e.g. |
|
|
Threshold for the minor cycle loop. Can be varied for each
polarisation. (e.g. |
|
|
|
The target peak residual. Major cycles stop if this is
reached. A negative number ensures all major cycles requested
are done. Can be given as an array with different values for
each pol loop (e.g. |
|
|
|
ncycles (cimager) |
Number of major cycles. Can be given as an array with
different values for each pol loop (e.g. |
|
false |
Images.writeAtMajorCycle (cimager) |
If true, the intermediate images will be written (with a .cycle suffix) after the end of each major cycle. |
|
MAXBASE |
Clean.solutiontype (see discussion at Multi-Scale and/or Multi-Frequency deconvolution) |
The type of peak finding algorithm to use in the deconvolution. Choices are MAXCHISQ, MAXTERM0, or MAXBASE. |
|
|
Clean.readscalemask (Solvers) |
If not blank, this is the name of the scalemask image. This functionality needs further development and so users should use this at their own risk! |
|
true |
none |
If true, only read the scale mask for Stokes I imaging. If false, it is read for all Stokes images. |
|
0 |
Clean.noiseboxsize(Solvers) |
Size of the box used to determine the local noise value for setting thresholds (when a “sigma” threshold is given). A value of 0 means the entire image is used. |
Optional Image products¶
Imager allows users to specify whether or not to write optional output products such as the residual, weights, natural psf, preconditioned psf etc. images. These are aimed at minimising disk I/O and occupancy where possible.
Note that information on weights is required by the linear mosaicking applications: linmos
and linmos-mpi. For non A-Projection gridders like WProject, where the weights value
across the image extent is constant, imager can write out the weight value
into an ascii text file that linmos can make use of. This can be specified using
WRITE_WEIGHTS_LOG_CONT
parameter for the continuum imaging case.
For A-project gridders, or when snapshot imaging is turned on, one will have to necessarily write out the weights images for use in linmos. An exception will be thrown by processASKAP.sh if this condition is violated.
More details on writing these optional image products are discussed in the sections for continuum and continuum cube imaging below. See also User Parameters - Spectral-line Imaging and User Parameters - Mosaicking.
Variable |
Default |
Parset equivalent |
Description |
---|---|---|---|
Optional outputs |
|||
|
true |
write.residualimage (imager) |
Whether to write the residual image |
|
false |
write.psfrawimage (imager) |
Whether to write the naturally weighted psf image |
|
false |
write.psfimage (imager) |
Whether to write the preconditioned psf image |
|
false |
write.weightsimage (imager) |
Whether to write the weights image |
|
true |
write.weightslog (imager) |
Option to write the constant weight into an ascii text file Note 1: Should not be used with A-project gridders. Note 2: For snapshot imaging, this option is internally forced
to “false” by the pipeline scripts. Set
|
|
true |
write.modelimage (imager) |
Whether to write the model image Note: The pipeline ensures that intermediate model images get generated if required by selfcal jobs. |
|
false |
write.maskimage (imager) |
Whether to write the mask image (continuum only) |
|
false |
write.scalemask (imager) |
Whether to write the scalemask image (continuum only) |
|
false |
write.sensitivityimage (imager) |
Whether to write sensitivity image (continuum only) |
|
true |
write.firstrestore (imager) |
Whether to write the first restored image when alt-restored images are requested (continuum only) |
Continuum cube imaging¶
Once the gains have been applied to the averaged dataset, it can be
imaged as “continuum cubes” in a variety of polarisations - use
DO_CONTCUBE_IMAGING=true
to turn this mode on.
This table lists all relevant parameters that are different to the MFS imaging. Some parameters, notably gridding and preconditioning, are the same. There are also parameters related to the use of imager.
Variable |
Default |
Parset equivalent |
Description |
---|---|---|---|
Imaging parameters |
|||
|
false |
none |
Whether to create continuum cubes |
|
|
none |
Time request for individual continuum cube jobs |
|
i.%t.SB%s.contcube |
Helps form Images.name (imager) |
Base name for the continuum cubes. It should include “i.”, as the actual base name will include the correct polarisation (‘I’ will produce i.contcube, Q will produce q.contcube and so on). The %s wildcard will be resolved into the scheduling block ID, and %t will be replaced by the “target” or scheduling block alias. |
|
4096 |
Images.shape (imager) |
Number of pixels on the spatial dimension for the continuum cubes. |
|
|
Images.cellsize (imager) |
Angular size of spatial pixels for the continuum cubes. If
not provided, it defaults to the value of |
|
|
Images.polarisation (imager) |
List of polarisations to create cubes for. This should be a comma-separated list of (upper-case) polarisations. Separate jobs will be launched for each polarisation given. |
|
|
Images.restFrequency (imager) |
Rest frequency to be written to the continuum cube. If left blank, no rest frequency is written. |
|
0 |
MaxUV (Data Selection) |
A maximum UV distance (in metres) to apply in the data selection step. Only used if a positive value is applied. |
|
0 |
MinUV (Data Selection) |
A minimum UV distance (in metres) to apply in the data selection step. Only used if a positive value is applied. |
|
true |
Images.nyquistgridding (imager) |
Whether to turn on Nyquist gridding. |
|
fit |
Restoring beam to use: ‘fit’ will fit the PSF in each channel
separately to determine the appropriate beam for that
channel, else give a size (such as
|
|
|
0.5 |
restore.beam.cutoff (imager) |
Cutoff value used in determining the support for the fitting (ie. the rectangular area given to the fitting routine). Value is a fraction of the peak. |
|
mid |
restore.beamReference (imager) |
Which channel to use as the reference when writing the restoring beam to the image cube. Can be an integer as the channel number (0-based), or one of ‘mid’ (the middle channel), ‘first’ or ‘last’ |
|
|
none |
Total number of cores to use for the continuum cube job. If
left blank, this will be chosen to match the number of
channels (taking into account |
|
3 (galaxy), 4 (petrichor), 6 (setonix) |
nchanpercore (imager) |
If imager (imager) is used, this determines how many channels each worker will process. |
|
8 (galaxy), 48 (petrichor), 73 (setonix) |
none |
How many of the cores on each node to use. |
Preconditioning |
|||
|
|
preconditioner.Names (Solvers) |
List of preconditioners to apply. If left blank, the preconditioning used for the continuum imaging is used instead, and any values given in the config file for the subsequent parameters are ignored. |
|
|
preconditioner.GaussianTaper (Solvers) |
Size of the Gaussian taper - either single value (for circular taper) or 3 values giving an elliptical size. |
|
|
preconditioner.GaussianTaper.isPsfSize (Solvers) |
Decide if rather than applying the specified taper, the taper should be adjusted to ensure that the output image planes have the specified resolution. |
|
0.005 |
preconditioner.GaussianTaper.tolerance (Solvers) |
Fractional tolerance for the fitted beam size when
|
|
-0.5 |
preconditioner.Wiener.robustness (Solvers) |
Robustness value for the Wiener filter. |
|
|
preconditioner.Wiener.taper (Solvers) |
Size of gaussian taper applied in image domain to Wiener
filter. Ignored if blank (ie. |
|
|
uvweight (imager) |
List of parameters for application of traditional weights. |
|
-0.5 |
uvweight.robustness (imager) |
The value of robustness. Only used if there is Robust among
the list of parameters given by
|
Continuum cube cleaning |
Different cleaning parameters used for the continuum cubes |
||
|
Clean |
Which solver to use. You will mostly want to leave this as ‘Clean’, but there is a ‘Dirty’ solver available. |
|
|
BasisfunctionMFS |
Clean.algorithm (Solvers) |
The name of the clean algorithm to use. |
|
false |
none |
Whether to use the MFS model image as the starting point for the cleaning of the continuum cube. |
|
600 |
Clean.niter (Solvers) |
The number of iterations for the minor cycle clean. |
|
0.2 |
Clean.gain (Solvers) |
The loop gain (fraction of peak subtracted per minor cycle). |
|
256 |
Clean.psfwidth (Solvers) |
The width of the psf patch used in the minor cycle. |
|
|
Clean.scales (Solvers) |
Set of scales (in pixels) to use with the multi-scale clean. |
|
|
threshold.minorcycle (Solvers) |
Threshold for the minor cycle loop. |
|
0.06mJy |
threshold.majorcycle (Solvers) |
The target peak residual. Major cycles stop if this is reached. A negative number ensures all major cycles requested are done. |
|
3 |
ncycles (imager) |
Number of major cycles. |
|
false |
Images.writeAtMajorCycle (imager) |
If true, the intermediate images will be written (with a .cycle suffix) after the end of each major cycle. |
|
MAXBASE |
Clean.solutiontype (see discussion at Multi-Scale and/or Multi-Frequency deconvolution) |
The type of peak finding algorithm to use in the deconvolution. Choices are MAXCHISQ, MAXTERM0, or MAXBASE. |
|
true |
Clean.detectdivergence (Solvers) |
Whether to detect that the deconvolution is starting to diverge (in which case it is stopped). |
|
true |
Clean.detectmilddivergence (Solvers) |
Whether to use the mild form of divergence detection, that should skip to the next major cycle if residuals increase by 10% or more. |
|
“[1.1,2.0,2.0,0.05]” |
Clean.divergencelevels (Solvers) |
Setting the levels for divergence detection to be used in the continuum-cube imaging. |
|
0 |
Clean.noiseboxsize(Solvers) |
Size of the box used to determine the local noise value for setting thresholds (when a “sigma” threshold is given). A value of 0 means the entire image is used. |
|
|
Clean.readscalemask (Solvers) |
If not blank, this is the name of the scalemask image. This functionality needs further development and so users should use this at their own risk! |
|
true |
none |
If true, only read the scale mask for Stokes I imaging. If false, it is read for all Stokes contcubes. |
New imager parameters |
|||
|
|
none |
If true, the continuum cube imaging is done by imager
(imager). If false, it is done by cimager
(cimager). When true, the following
parameters are used. If left blank (the default), the value
is given by the overall parameter |
|
12 (galaxy/petrichor), 24 (setonix) |
nchanpercore (imager) |
The number of channels each core will process. |
|
|
nwriters (imager) |
The number of writers used by imager. Unless
|
|
true |
singleoutputfile (imager) |
Whether to write a single cube, even with multiple writers
(ie. |
|
true |
none |
Whether to mask out bad or blank channels from the continuum |
|
false |
none |
Whether to use the “signifcance”, or the ratio of the 1-percent statistic to the MADFM, to determine the bad channels. |
|
none |
The significance level at which to reject channels. |
|
|
true |
none |
Whether to mask out bad channels on the basis of the MADFM value alone. |
|
none |
The value of MADFM (in mJy/beam), above which a channel is deemed bad. |
|
|
true |
none |
Whether to mask out blank channels from the continuum cube. |
Optional outputs |
|||
|
true |
write.residualimage (imager) |
Whether to write the residual image cube |
|
false |
write.psfrawimage (imager) |
Whether to write the naturally weighted psf image cube |
|
false |
write.psfimage (imager) |
Whether to write the preconditioned psf image cube |
|
false |
write.weightsimage (imager) |
Whether to write the weights image cube |
|
true |
write.weightslog (imager) |
Option to write the weight spectra into an ascii text file Note 1: Should not be used with A-project gridders. Note 2: For snapshot imaging, this option is internally forced
to “false” by the pipeline scripts. Set
|
|
true |
write.modelimage (imager) |
Whether to write the model image cube Note: The pipeline ensures that intermediate model images get generated if required by selfcal jobs. |
|
false |
write.grids (imager) |
Whether to write the UV-grids (cubes only) |
Fast / Transient imaging¶
The pipeline presents a transient-imaging workflow, where continuum images are made over intervals shorter than the total observation time. The workflow is summarised as follows:
For a given beam dataset, the deep continuum clean model (from the full continuum imaging) is subtracted using
ccontsubtract
, to create a coarse-resolution continuum-subtracted MS.This MS is imaged sequentially in each of a number of time intervals spanning the full observation duration:
The intervals are calculated by the python script
transientIntervals.py
. This takes a nominated interval length (TRANSIENT_INTERVAL
, in seconds), and adjusts it to be a whole number of integrations.A series of start and end times are then calculated, which are presented to
imager
via the parset parameterCimager.TimeRange
(see imager).Each interval is then imaged in sequence, producing a single restored image for each interval.
These images are then given to the Vaster module, a containerised set of scripts provided by the VAST Survey Science Team, to perform transient detection. This uses the deep component catalogue for the beam (to determine locations to search), and so the per-beam sourcefinding is activated when transient imaging is turned on.
This results in a tar.gz file containing the Vaster outputs, that can be copied elsewhere for immediate use (a more complete archiving scheme is under development).
This is all done separately for each beam, in a specific transientImaging subdirectory.
The imaging is much simpler than the other continuum imaging. Since the dataset has already had the bulk of the continuum emission removed, only a very simple, shallow clean is done. Some parameters are provided to alter the preconditioning or deconvolution (see table below), although it is anticipated the defaults should suffice. The image can be made a different size/shape to the main continuum images, and no MFS imaging is done by default.
Variable |
Default |
Parset equivalent |
Description |
---|---|---|---|
|
false |
none |
Whether to run the transient imaging & detection |
|
900 |
none |
Length of time in seconds for a single interval to be imaged. |
Imaging parameters |
|||
|
3584 |
Images.shape (imager) |
Number of pixels on the spatial dimension for the transient images |
|
|
Images.cellsize(imager) |
Angular size of spatial pixels for the transient images. If
not provided, it defaults to the value of |
|
1 |
ncycles (imager) |
Number of major cycles |
|
BasisfunctionMFS |
Clean.algorithm (Solvers) |
The name of the clean algorithm to use. |
|
200 |
Clean.niter (Solvers) |
The number of iterations for the minor cycle clean. |
|
“[30%, 5sigma, 0.5sigma]” |
threshold.minorcycle (Solvers) |
Thresholds for the minor cycle loop |
|
0.2 |
Clean.gain (Solvers) |
The loop gain (fraction of peak subtracted per minor cycle). |
|
256 |
Clean.psfwidth (Solvers) |
The width of the psf patch used in the minor cycle |
|
“[0]” |
Clean.scales (Solvers) |
Set of scales (in pixels) to use with the multi-scale clean. |
|
0 |
Clean.noiseboxsize(Solvers) |
Size of the box used to determine the local noise value for setting thresholds (when a “sigma” threshold is given). A value of 0 means the entire image is used. |
|
“[Wiener]” |
preconditioner.Names (Solvers) |
List of preconditioners to apply. |
|
-0.5 |
preconditioner.Wiener.robustness (Solvers) |
Robustness value for the Wiener filter. |
|
|
preconditioner.Wiener.taper (Solvers) |
Size of gaussian taper applied in image domain to Wiener
filter. Ignored if blank (ie. |
|
“[10arcsec, 10arcsec, 0deg]” |
preconditioner.GaussianTaper (Solvers) |
Size of the Gaussian taper (if provided in the preconditioner list) - either single value (for circular taper) or 3 values giving an elliptical size. |
|
0.005 |
preconditioner.GaussianTaper.tolerance (Solvers) |
Fractional tolerance for the fitted beam size when
|
|
false |
preconditioner.GaussianTaper.isPsfSize (Solvers) |
Decide if rather than applying the specified taper, the taper should be adjusted to ensure that the output image planes have the specified resolution. |
|
|
uvweight (imager) |
List of parameters for application of traditional weights. |
|
-0.5 |
uvweight.robustness (imager) |
The value of robustness. Only used if there is Robust among
the list of parameters given by
|
Transient Detection |
|||
|
“chisquare, peak, std” |
none |
Input parameters for Vaster |
|
4 |
none |
Input parameter for Vaster |
|
100 |
none |
Input parameter for Vaster |
|
100 |
none |
Input parameter for Vaster |
Data handling |
|||
|
“” |
none |
Location to copy the output tar.gz files to - should be a directory. Leaving blank will keep the output files in the transientImaging directory. |