imager ====================================================================== This page provides some details about the imager program. The purpose of this software is to perform imaging in a parallel/distributed environment or on a single computer system. The software leverages MPI, however can be run on a simple laptop or a large supercomputer. It provides all the functionality of the cimager program but permits a more flexible distribution of data and has a smaller aggregated memory footprint. By default the imager processes each channel independently (grid/FFT) and in continuum mode all channels are combined in image space. Although beneficial for spectral line data-sets, especially those with single channel tiling. It is not optimal for continuum imaging with many-channel tiling as it performs much more IO and nchannel x more FFTs than are required. Therefore if imager is processing in continuum mode it is suggested that the following option be used:: Cimager.combinechannels = true This switches on a mode where the multiple channels of a machine allocation are accessed and gridded together before the FFT. This improves IO performance and reduces the FFT count roughly by the number of channels in each allocation (see nchanpercore). Note this is by default. Running the program ------------------- It can be run with the following command, where "config.in" is a file containing the configuration parameters described in the next section. :: $ imager -c config.in The config file is the same format as that required by the *cimager* this has been done to accentuate the similarity between the two imagers. Both the *Cimager* or *imager* prefix are accepted, but you can't mix the two in one config file. Parallel/Distributed Execution ------------------------------ The program is distributed and used a master/worker pattern to distribute and manage work. The program requires at least two processes to execute, and failure to either execute *imager* as an MPI process or specifying only one MPI process will result in an error. On the Cray EX platform executing with MPI takes the form:: $ srun -n 289 -N 4 imager -c config.in The *-n* and *-N* parameters to the *srun* application launcher specify 289 MPI processes will be used (288 workers and one master) distributed over 4 compute nodes, each node will host 72 or 73 MPI processes. As with *cimager* the *imager* has no implicit parallelism, rather the configuration parameters allow the user to specify what subset of the total task each worker will carry out. This is controlled by the nchanpercore keyword. **Allocation of work and temporary files** It should be noted that *imager* can be run in exactly the manner as *cimager* if required, but its primary benefit is that you do not have to. The caveat is that imager always needs at least two processes to run. **Example 1: Multiple Channels Per Core** Typically, if we are processing a single measurement set with 288 spectral channels, using 289 workers the following configuration parameter can be used to restrict each worker to a single spectral channel:: Cimager.nchanpercore = 1 You will need 289 cores to run. However if you decide that a core will process 4 channels. Then you would use:: Cimager.nchanpercore = 4 Now you would only need to invoke:: $ srun -n 73 -N 1 imager -c config.in This would only require 1 node. **The Channels and Frequencies keywords**: The Channels keyword behaves in a similar way to cimager: Cimager.Channels = [,] You can specify the number of channels and the start channel. This will be taken from the complete list of channels as determined by the input measurement sets. The Frequencies keyword that allows you to specify the dimensions of the spectral cube in frequency space: Cimager.Frequencies = [,,] - all in Hz (This keyword has not been tested extensively with the freqframe options) **Update Specifying the Frequency frame** Cimager.freqframe = topo | bary | lsrk You can choose a default frequency frame for the spectral cube as either, topocentric (the default), barycentric or the kinematic local standard of rest. Other frames can be added on request. **Example 2: Beam Selection** This imager has built in splitting of the input measurement set. At the moment only a single beam is split out but it can be any one and is determined by the first entry int the following vector:: Cimager.beams =[0] **Note: The Local Merge** The minor cycle of clean is currently still performed on a single master core. But the individual channel allocations are merged locally to the worker core. Spectral-line processing ------------------------ Imager can also be used for spectral line processing. Each channel is processed independently and written to one or more shared output cubes. To enable this mode use:: Cimager.solverpercore = true **Example 3: Multiple writers** When writing CASA images this imager can write to mode than one output image cube to improve disk throughput. This has been implemented to remove a serious bottle neck in the spectral line processing. For FITS imagetypes a single cube can be written - but multiple writers are used - once again to improve write performance:: Cimager.nwriters = 16 Cimager.singleoutputfile = true **Example 4: Joint cleaning mode / per beam processing** In spectral line mode only, imager can be instructed to process each measurement set independently and to merge the subimage into a larger image for the minor cycles. Note this is different to faceting - which processes the sub images entirely independently. This scheme grids and images the fields individually but cleans them jointly as in Cornwell 1989. You can specify the shape of the mosaicked image and the subshape of the individual images:: Cimager.updatedirection = true Cimager.solverpercore = true Cimager.Images.shape = [4096,4096] Cimager.Images.subshape = [2048,2048] Each subimage will be centred on its tangent point as determined by the MeasurementSet and selection, but the centre of the overall mosaic image can be set using the image specific direction parameter:: Cimager.Images.*Imagename*.direction = [12h30m00.00, -45.00.00.00, J2000] Nyquist gridding ---------------- To decrease memory usage and runtime, visibilities can be gridded onto the smallest uv grid that can fit them all. This grid size is used for all uv grids, for the merge and main image FFT operations, and for preconditioning. FFT padding is then used before cleaning and image output to set the desired *cellsize* and *shape*. This feature is enabled using either of the two relevant parameters defined below: *griddingcellsize* or *nyquistgridding*. **Example 5: Memory reduction** For SB11816 the largest uvw coordinates are 28822.3, 29641.3 and 24250.9 wavelengths. So an ideal cellsize that oversamples the synthesised beam by a factor of six is around 1.16 arcseconds. But that resolution is not needed until cleaning. Before then grids can retain all of the information with 2x oversampling. The *nyquistgridding* parameter determines a *griddingcellsize* of 3.25arcsec:: Cimager.Images.shape = [6144, 6144] Cimager.Images.cellsize = [1.16arcsec, 1.16arcsec] Cimager.Images.nyquistgridding = true Or alternatively the coarse resolution can be entered directly:: Cimager.Images.griddingcellsize = [3.25arcsec, 3.25arcsec] Using the BasisfunctionMFS solver with four scales and two Taylor terms, the memory usage reported when running on nine worker nodes is: PeakVM (MB): +--------+-----+----+----+----+----+----+----+----+----+----+ |Rank | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | +========+=====+====+====+====+====+====+====+====+====+====+ |Normal |14432|6397|5326|5324|5724|4727|4715|4681|4392|4392| +--------+-----+----+----+----+----+----+----+----+----+----+ |Nyquist | 9166|3132|2826|2815|2907|2739|2724|2724|2632|2632| +--------+-----+----+----+----+----+----+----+----+----+----+ PeakRSS (MB): +--------+-----+----+----+----+----+----+----+----+----+----+ |Rank | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | +========+=====+====+====+====+====+====+====+====+====+====+ |Normal |11976|3044|3048|3041|2447|2455|2446|2132|2132|2131| +--------+-----+----+----+----+----+----+----+----+----+----+ |Nyquist | 6997| 922| 922| 910| 850| 849| 849| 795| 758| 758| +--------+-----+----+----+----+----+----+----+----+----+----+ Configuration Parameters ------------------------ Parset parameters understood by imager are given in the following table (all parameters must have either **Cimager** or **imager** prefix, i.e. **Cimager.dataset**). For a number of parameters certain keywords are substituted, i.e. **%w** is replaced by the worker number (rank-1, if there is only one pool of workers) and **%n** by the number of nodes in the parallel case. In the serial case, these special strings are substituted by 0 and 1, respectively. This substitution allows to reuse the same parameter file on all nodes of the cluster if the difference between jobs assigned to individual nodes can be coded by using these keywords (e.g. using specially crafted file names). Note, if there is more than 1 group of workers (e.g. parallel calculation of Taylor terms), %w index spans the workers in one group rather than the global pool of workers. This is done to allow the same file name to be used for corresponding worker in different groups (i.e. all Taylor terms are build from the same file). If a parameter supports substitution, it is clearly stated in the description. A number of other parameters allowing to narrow down the data selection are understood. They are given in a separate table (see :doc:`data_selection`) and should also have the **Cimager** prefix. Note that option **Cimager.CorrelationType** will be ignored and default to "cross". To record the individual channel beams when run in spectral-line mode, the imager can produce an ASCII text file listing the beam parameters for each channel. This is known as the "beam log". If the image cube name is "image.i.blah", then the beam log will be called "beamlog.image.i.blah.txt". The file has columns: index | major axis[arcsec] | minor axis [arcsec] | position angle [deg] Should the imaging of a channel fail for some reason, the beam for that channel will be recorded as having zero for all three parameters. This beam log is compatible with other askapsoft tasks, specfically the spectral extraction in Selavy (see :doc:`../analysis/extraction`). Here is an example of the start of a beam log:: #Channel BMAJ[arcsec] BMIN[arcsec] BPA[deg] 0 64.4269 59.2985 -70.8055 1 64.4313 59.299 -70.8831 2 64.4333 59.3018 -70.9345 3 64.4338 59.2996 -70.9256 4 64.4349 59.2982 -70.9108 Similar logs can be produced for per channel weights and image statistics. Alternatively, these data can be written to a subtable (casa format) or binary table (FITS format) in the image file. +--------------------------+------------------+--------------+----------------------------------------------------+ |**Parameter** |**Type** |**Default** |**Description** | +==========================+==================+==============+====================================================+ |imagetype |string |"casa" |Type of the image handler (determines the format of | | | | |the images, both which are written to or read from | | | | |the disk). The default is to create casa images but | | | | |"fits" can also be chosen. | +--------------------------+------------------+--------------+----------------------------------------------------+ |imagealloc |string |"fast" |Set to "fast" to use the fast file allocation scheme| | | | |This can save a lot of time when creating large | | | | |cubes. This is the default, use "" to turn off. | +--------------------------+------------------+--------------+----------------------------------------------------+ |dataset |string or |None |Measurement set file name to read from. Usual | | |vector | |substitution rules apply if the parameter is a | | | | |single string. If the parameter is given as a vector| | | | |then the sets can be different frequencies of the | | | | |same observation or different epochs | +--------------------------+------------------+--------------+----------------------------------------------------+ |nworkergroups |int |1 |Number of worker groups. This option can only be | | | | |used in the parallel mode. If it is greater than 1, | | | | |the model parameters are distributed (as evenly as | | | | |possible) between the given number of groups of | | | | |workers (e.g. if one calculates a Taylor term | | | | |expansion of the order of 1 for one image, setting | | | | |this parameter to 3 will allow parallel computation | | | | |of the Taylor terms for this image). This is on top | | | | |of the normal parallelism within the group (the %w | | | | |index spans from 0 to the number of workers per | | | | |group - 1). Essentially, this option allows to throw| | | | |several workers on the same problem if the model | | | | |allows partitioning.Taylor terms, faceting and | | | | |multiple images in the model are the typical use | | | | |cases. | +--------------------------+------------------+--------------+----------------------------------------------------+ |nchanpercore |int |1 |Number of channels allocated to each worker core | | | | | | +--------------------------+------------------+--------------+----------------------------------------------------+ |combinechannels |bool |false |When true combine all nchanpercore channels into a | | | | |single grid before the FFT. Generally optimal for | | | | |continuum (Taylor term) processing. | +--------------------------+------------------+--------------+----------------------------------------------------+ |Channels |vector |[] |Channels to be selected from the measurement set. | | | | |Syntax is [,]. Defaults to all the | | | | |channels | +--------------------------+------------------+--------------+----------------------------------------------------+ |Frequencies |vector |[] |Dimensions of the output cube in freuency space | | | | |Syntax is [,, |[0] |Beam number to be selected from the measurement set | | | | | | +--------------------------+------------------+--------------+----------------------------------------------------+ |nwriters |int |1 |The number of output cubes to | | | | |generate in spectral cube mode. | +--------------------------+------------------+--------------+----------------------------------------------------+ |freqframe |string |topo |Generate output cubes in the given frame | | | | |options are topocentric (topo), barycentric (bary) | | | | |and kinematic local standard of rest (lsrk) | +--------------------------+------------------+--------------+----------------------------------------------------+ |dopplertracking |bool |false |For use with bary and lsrk frame: track the change | | | | |in frequency during the observation, selecting the | | | | |nearest channel at each time. Note that by default | | | | |this uses the field direction (antenna pointing) | | | | |as direction reference, not the tangent point or | | | | |beam pointing | +--------------------------+------------------+--------------+----------------------------------------------------+ |dopplertracking.direction |vector |None |Specify the reference direction to use for the | | | | |velocity calculations, | | | | |e.g., [12:34:56.7,-23.45.56.7,J2000] | +--------------------------+------------------+--------------+----------------------------------------------------+ |singleoutputfile |bool |false |Single output cube. Useful in the case of multiple | | | | |writers | +--------------------------+------------------+--------------+----------------------------------------------------+ |solverpercore |bool |false |Turn on distributed solver (spectral imaging) mode | +--------------------------+------------------+--------------+----------------------------------------------------+ |updatedirection |bool |false |Turn on joint cleaning mode (see above) | +--------------------------+------------------+--------------+----------------------------------------------------+ |datacolumn |string |"DATA" |The name of the data column in the measurement set | | | | |which will be the source of visibilities.This can be| | | | |useful to process real telescope data which were | | | | |passed through *casapy* at some stage (e.g. to image| | | | |calibrated data which are stored in the | | | | |*CORRECTED_DATA* column). In the measurement set | | | | |convention, the *DATA* column which is used by | | | | |default contains raw uncalibrated data as received | | | | |directly from the telescope. Calibration tasks in | | | | |*casapy* make a copy when calibration is applied | | | | |creating a new data column. | +--------------------------+------------------+--------------+----------------------------------------------------+ |sphfuncforpsf |bool |false |If true, the default spheroidal function gridder is | | | | |used to compute PSF regardless of the gridder | | | | |selected for model degridding and residual | | | | |gridding. This has a potential to produce better | | | | |behaving PSF by taking out two major factors of | | | | |position dependence. Note, this doesn't make the PSF| | | | |correct or otherwise,it is just a different | | | | |approximation | +--------------------------+------------------+--------------+----------------------------------------------------+ |calibrate |bool |false |If true, calibration of visibilities will be | | | | |performed before imaging. See | | | | |:doc:`calibration_solutions` for details on | | | | |calibration parameters used during this application | | | | |process. | +--------------------------+------------------+--------------+----------------------------------------------------+ |calibrate.scalenoise |bool |false |If true, the noise estimate will be scaled in | | | | |accordance with the applied calibrator factor to | | | | |achieve proper weighting. | +--------------------------+------------------+--------------+----------------------------------------------------+ |calibrate.allowflag |bool |false |If true, corresponding visibilities are flagged if | | | | |the inversion of Mueller matrix fails. Otherwise, an| | | | |exception is thrown should the matrix inversion fail| +--------------------------+------------------+--------------+----------------------------------------------------+ |calibrate.ignorebeam |bool |false |If true, the calibration solution corresponding to | | | | |beam 0 will be applied to all beams | +--------------------------+------------------+--------------+----------------------------------------------------+ |calibrate.interpolatetime |bool |false |If true, do linear interpolation of gain solutions | +--------------------------+------------------+--------------+----------------------------------------------------+ |gainsfile |string |"" |This is an obsolete parameter, which is still | | | | |supported for backwards compatibility defining the | | | | |file with antenna gains (a parset format, keywords | | | | |look like **gain.g11.0**, where g11 or g22 in the | | | | |middle correspond to different polarisations and the| | | | |trailing number is the zero-based antenna | | | | |number. The default value (empty string) means no | | | | |gain correction is performed. The gain file format | | | | |is the same as produced by Ccalibrator. | +--------------------------+------------------+--------------+----------------------------------------------------+ |restore |bool |false |If true, the image will be restored (by convolving | | | | |with the given 2D gaussian). This is an additional | | | | |step to normal imaging, which, by default, ends with| | | | |just a model image. The restored image is written | | | | |into a separate image file (with the **.restore** | | | | |suffix). The convolution is done with the restore | | | | |solver (see also :doc:`solver`) which reuses the | | | | |same parameters used to setup the image solver (and | | | | |therefore ensuring the same preconditioning is | | | | |done). The only additional parameter of the restore | | | | |solver is the shape of the gaussian representing | | | | |clean beam (or flag to determine the shape). It is | | | | |given by the **restore.beam** parameter, which must | | | | |be present if **restore** is set to True | +--------------------------+------------------+--------------+----------------------------------------------------+ |residuals |bool |false |If true write out the residual image. (deprecated) | +--------------------------+------------------+--------------+----------------------------------------------------+ |restore.beam |vector |None |Either a single word *fit* or a quantity string | | | | |describing the shape of the clean beam (to convolve | | | | |the model image with). If quantity is given it must | | | | |have exactly 3 elements, e.g. [30arcsec, 10arcsec, | | | | |40deg]. Otherwise an exception is thrown. This | | | | |parameter is only used if *restore* is set to | | | | |True. If restore.beam=fit, the code will fit a 2D | | | | |gaussian to the PSF image (first encountered if | | | | |multiple images are solved for) and use the results | | | | |of this fit. | +--------------------------+------------------+--------------+----------------------------------------------------+ |restore.beam.cutoff |double |0.5 |Cutoff for the support search prior to beam fitting,| | | | |as a fraction of the PSF peak. This parameter is | | | | |only used if *restore.beam=fit*. The code does | | | | |fitting on a limited support (to speed things up and| | | | |to avoid sidelobes influencing the fit). The extent | | | | |of this support is controlled by this parameter | | | | |representing the level of the PSF which should be | | | | |included into support. This value should be above | | | | |the first sidelobe level for meaningful results. | +--------------------------+------------------+--------------+----------------------------------------------------+ |restore.beam.maxsupport |int |101 |Maximum support size for beam fitting - this avoids | | | | |spending excessive CPU time trying to fit very poor | | | | |beams | +--------------------------+------------------+--------------+----------------------------------------------------+ |restore.preconditioner.xxx|vector |None |Specify an additional preconditioner to be used | | | | |when restoring. This will result in a second set | | | | |of restored files with the *.alt.restored* suffix. | | | | |See **preconditioner** below. When using | | | | |multiple restore solvers, one should use the | | | | |**restore.beam=fit** option to allow for different | | | | |beam shapes. | +--------------------------+------------------+--------------+----------------------------------------------------+ |restore.preconditioner |string |"alt" |Change the suffix for the alternative preconditioner| |.suffix | | |products | +--------------------------+------------------+--------------+----------------------------------------------------+ |restore.equalise |bool |false |If true, the final residual is multiplied by the | | | | |square root of the truncated normalised weight | | | | |(i.e. additional weight described by Sault et | | | | |al. (1996), which gives a flat noise). Note, that | | | | |the source flux densities are likely to have | | | | |position-dependent errors if this option is used | | | | |because not all flux is recovered during the clean | | | | |process. However, the images look aesthetically | | | | |pleasing with this option. | +--------------------------+------------------+--------------+----------------------------------------------------+ |restore.updateresiduals |bool |true |The residual image written out by the restore solver| | | | |can be updated using the latest model. This is now | | | | |the default behviour. Note the majorcycle outputs do| | | | |not pass through the restore solver so are not | | | | |updated so therefore correspond to the residuals at | | | | |the beginning of the last minor cycle. | +--------------------------+------------------+--------------+----------------------------------------------------+ |Images.xxx |various | |A number of parameters given in this form define the| | | | |images one wants to produce (shapes, positions, | | | | |etc). The details are given in a separate section | | | | |(see below) | +--------------------------+------------------+--------------+----------------------------------------------------+ |nUVWMachines |int32 |number of |Size of uvw-machines cache. uvw-machines are used to| | | |beams |convert uvw from a given phase centre to a common | | | | |tangent point. To reduce the cost to set the machine| | | | |up (calculation of the transformation matrix), a | | | | |number of these machines is cached. The key to the | | | | |cache is a pair of two directions: the current phase| | | | |centre and the tangent centre. If the required pair | | | | |is within the tolerances of that used to setup one | | | | |of the machines in the cache, this machine is | | | | |reused. If none of the cache items matches the least| | | | |accessed one is replaced by the new machine which is| | | | |set up with the new pair of directions. The code | | | | |would work faster if this parameter is set to the | | | | |number of phase centres encountered during | | | | |imaging. In non-faceting case, the optimal setting | | | | |would be the number of synthetic beams times the | | | | |number of fields. For faceting (btw, the performance| | | | |gain is quite significant in this case), it should | | | | |be further multiplied by the number of | | | | |facets. Direction tolerances are given as a separate| | | | |parameter. | +--------------------------+------------------+--------------+----------------------------------------------------+ |uvwMachineDirTolerance |quantity string |"1e-6rad" |Direction tolerance for the management of the | | | | |uvw-machine cache (see *nUVWMachines* for | | | | |details). The value should be an angular | | | | |quantity. The default value corresponds roughly to | | | | |0.2 arcsec and seems sufficient for all practical | | | | |applications within the scope of ASKAPsoft. | +--------------------------+------------------+--------------+----------------------------------------------------+ |gridder |string |None |Name of the gridder, further parameters are given by| | | | |*gridder.something*. See :doc:`gridder` for details.| | | | | | +--------------------------+------------------+--------------+----------------------------------------------------+ |includeflagged |bool |true |By default the flags are ignored when scanning | | | | |through the data for ranges of frequencies, uvw | | | | |values and other parameters because this is faster. | | | | |Set to false to exclude flagged data and make the | | | | |advice more accurate. Flagged data are always | | | | |excluded during imaging. | +--------------------------+------------------+--------------+----------------------------------------------------+ |rankstoringcf |int |1 |In the parallel mode, only this rank will attempt to| | | | |export convolution functions if this operation is | | | | |requested (see *tablename* option in the | | | | |:doc:`gridder`) This option is ignored in the serial| | | | |mode. | +--------------------------+------------------+--------------+----------------------------------------------------+ |visweights |string |"MFS" if any |If this parameter is set to "MFS" gridders are setup| | | |*nterms>1*, |to grid/degrid with the weight required for | | | |"" otherwise |multi-frequency synthesis. At the moment, this | | | | |parameter is decoupled from the image setup, which | | | | |has to be done separately in a consistent way to use| | | | |MSMFS (*nterms* should be set to something greater | | | | |than 1). | +--------------------------+------------------+--------------+----------------------------------------------------+ |visweights.MFS.reffreq |double |ave freq (see |Reference frequency in Hz for MFS processing (see | | | |*frequency* |above) | | | |above) | | +--------------------------+------------------+--------------+----------------------------------------------------+ |solver |string |None |Name of the solver, further parameters are given by | | | | |*solver.something*. See :doc:`solver` for details | | | | | | +--------------------------+------------------+--------------+----------------------------------------------------+ |thershold.xxx |various | |Thresholds for the minor and major cycle (cycle | | | | |termination criterion), see :doc:`solver` for | | | | |details. | +--------------------------+------------------+--------------+----------------------------------------------------+ |preconditioner.xxx |various | |Preconditioners applied to the normal equations | | | | |before the solver is called, see :doc:`solver` for | | | | |details. | +--------------------------+------------------+--------------+----------------------------------------------------+ |ncycles |int32 |0 |Number of major cycles (and iterations over the | | | | |dataset) | +--------------------------+------------------+--------------+----------------------------------------------------+ |sensitivityimage |bool |false |If true, an image with theoretical sensitivity will | | | | |be created in addition to weights image (deprecated)| +--------------------------+------------------+--------------+----------------------------------------------------+ |sensitivityimage.cutoff |float |0.01 |Desired cutoff in the sensitivity image | +--------------------------+------------------+--------------+----------------------------------------------------+ |freqframe |string |topo |Frequency frame to work in (the frame is converted | | | | |when the dataset is read). Either lsrk or topo is | | | | |supported. | +--------------------------+------------------+--------------+----------------------------------------------------+ |channeltolerance |double |0 |Whether to use the floating-point tolerance in | | | | |comparing frequencies from different datasets, | | | | |allowing for small differences in the frequency | | | | |settings. Default is to require the frequencies to | | | | |match exactly. | +--------------------------+------------------+--------------+----------------------------------------------------+ |calcstats |bool |false |If true, calculate the image statistics and write it| | | | |to the image table. | +--------------------------+------------------+--------------+----------------------------------------------------+ |clearcache |bool |false |If true, clear the table cache for the DATA and FLAG| | | | |after processing each channel. This reduces memory | | | | |use for the spectral line case | +--------------------------+------------------+--------------+----------------------------------------------------+ Parameters of images ```````````````````` This section describes parameters used to define images, i.e. what area of the sky one wants to image and how. All parameters given in the following table have **Cimager.Images.\*** prefix, e.g. Cimager.Images.reuse = false. Parameters are global, affecting all images in the *Cimager.Images.Names* vector, unless the prefix includes one of those names, i.e. **Cimager.Images.ImageName.\***. In continuum mode you can specify more than one set of image parameters (like in Cimager) and all the images will be cleaned together with separate minor cycles but a combined major cycle. This can be used, e.g., to model and subtract sources outside the main image. The offset images can be made smaller than the main image to reduce memory usage. If Nyquist gridding is used all image shapes need to square and be integer multiples of the smallest image - they will be adjusted if this is not the case. In spectral mode (*solverpercore=true*) only a single image parameter is accepted and you should specify the default image parameters instead of the image specific ones - the exception is the joint cleaning mode (per beam processing) with *updatedirection=true* where the image specific parameter is used to specify the centre of the overall image. +--------------------------+----------------+-----------------------+----------------------------------------------+ |**Parameter** |**Type** |**Default** |**Description** | +==========================+================+=======================+==============================================+ |reuse |bool |false |If true, the model images will be read from | | | | |the disk (from the image files they are | | | | |normally written to according to the parset) | | | | |before the first major cycle. If false (the | | | | |default), a new empty model image will be | | | | |initialised for every image solved | | | | |for. Setting this parameter to true allows to | | | | |continue cleaning the same image if more major| | | | |cycles are required after inspection of the | | | | |image. Note, there is little cross check that | | | | |the image given as an input is actually a | | | | |result of the previous run of cimager with the| | | | |same Image parameters. So the user is | | | | |responsible to ensure that the projection, | | | | |shape, etc matches. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |shape |vector |1.7 * pb FWHM (~1st |Optional parameter to define the default shape| | | |null) + 2 * max(pb |for all images. If an individual *shape* | | | |offset) |parameter is specified separately for one of | | | | |the images, this default value of the shape is| | | |pb FWHM = 1.2*lambda/12|overridden. Individual *shape* parameters (see| | | | |below) must be given for all images if this | | | | |parameter is not defined. Must be a | | | | |two-element vector. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |cellsize |vector |1/max(u,v) / 6 rad |Optional parameter to define the default pixel| | | | |(or cell) size for all images. If an | | | | |individual *cellsize* parameter is specified | | | | |separately for one of the images, this default| | | | |value is overridden. Individual *cellsize* | | | | |parameters (see below) must be given for all | | | | |images, if this parameter is omitted. If | | | | |defined, a 2-element quantity string vector is| | | | |expected, e.g. [6.0arcsec, 6.0arcsec]. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |griddingcellsize |vector |None |Optional parameter to define the pixel size | | | | |used during gridding, merging and | | | | |preconditioning. Before cleaning or output, | | | | |images are oversampled to the desired | | | | |*cellsize*. | | | | |If defined, a 2-element quantity string vector| | | | |is expected, e.g. [6.0arcsec, 6.0arcsec]. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |nyquistgridding |bool |false |Optional parameter to determine the | | | | |*griddingcellsize* parameter automatically | | | | |using uvw metadata. If *griddingcellsize* is | | | | |also defined, that value will be used. Setting| | | | |nyquistgridding=false will not disable | | | | |*griddingcellsize* if it is defined. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |writeAtMajorCycle |bool |false |If true, the current images are written to | | | | |disk after each major cycle (*.cycle* suffix | | | | |is added to the name to reflect which major | | | | |cycle the image corresponds to). By default, | | | | |the images are only written after *ncycles* | | | | |major cycles are completed. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |Names |vector |None |List of image names which this imager will | | | | |produce. If more than one image is given, a | | | | |superposition is assumed (i.e. visibilities | | | | |are fitted with a combined effect of two | | | | |images; two measurement equations are simply | | | | |added). Parameters of each image defined in | | | | |this list must be given in the same parset | | | | |using *ImageName.something* keywords (with | | | | |usual prefix). Note, all image names must | | | | |start with word *image* (this is how | | | | |parameters representing images are | | | | |distinguished from other type of free | | | | |parameters in ASKAPsoft), otherwise an | | | | |exception is thrown. Example of valid names | | | | |are: *image.10uJy*, *image*, *imagecena* | | | | |In spectral mode only the first name is used. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.nchan |int32 |1 |Number of spectral planes in the image cube to| | | | |produce. Set it to 1 if just a 2D image is | | | | |required | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.frequency |vector |[min freq,max freq] if |Frequencies in Hz of the first and the last | | | |nchan>1, |spectral channels to produce in the cube. The | | | |[ave freq,ave freq] if |range is binned into *nchan* channels and the | | | |nchan=1 |data are gridded (with MFS) into a nearest | | | | |image channel (therefore, the number of image | | | |ave freq = (min+max)/2 |channels given by the *nchan* keyword may be | | | | |less than the number of spectral channels in | | | |Note: these are the |the data. If *nchan* is 1 all data are MFS'ed | | | |min and max |into a single image (however the image will | | | |frequencies being |have a degenerate spectral axis with the | | | |processed, which may |frequency defined by the average of the first | | | |be a subset of the |and the last element of this vector; it is | | | |full frequency range. |practical to make both elements identical, | | | | |when *nchan* is 1). The vector should contain | | | | |2 elements at all times, otherwise an | | | | |exception is thrown | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.direction |vector |phase centre of the |Direction to the centre of the required image | | | |visibilities |(or tangent point for facets). This vector | | | | |should contain a 3-element direction quantity | | | | |containing right ascension, declination and | | | | |epoch, e.g. [12h30m00.00, -45.00.00.00, | | | | |J2000]. Note that a casa style of declination | | | | |delimiters (dots rather than colons) is | | | | |essential. Only *J2000* directions are | | | | |currently supported. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.tangent |vector |"" |Direction to the user-defined tangent point, | | | | |if different from the centre of the | | | | |image. This vector should contain a 3-element | | | | |direction quantity containing right ascension,| | | | |declination and epoch, e.g. [12h30m00.00, | | | | |-45.00.00.00, J2000] or be empty (in this case| | | | |the tangent point will be in the image | | | | |centre). Note that a casa style of declination| | | | |delimiters (dots rather than colons) is | | | | |essential. Only *J2000* directions are | | | | |currently supported. This option doesn't work | | | | |with faceting. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.ewprojection |bool |false |If true, the image will be set up with the NCP| | | | |or SCP projection appropriate for East-West | | | | |arrays (w-term is equivalent to this | | | | |coordinate transfer for East-West arrays) | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.shape |vector |None |Optional parameter if the default shape | | | | |(without image name prefix) is defined. This | | | | |value will override the default shape for this| | | | |particular image. Must be a 2-element vector. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.cellsize |vector |None |Optional parameter if the default cell size | | | | |(without image name prefix) is defined. This | | | | |value will override the default cell size for | | | | |this particular image. A two-element vector of| | | | |quantity strings is expected, e.g. [6.0arcsec,| | | | |6.0arcsec] | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.nfacets |int32 |1 |Number of facets for the given image. If | | | | |greater than one, the image centre is treated | | | | |as a tangent point and *nfacets* facets are | | | | |created for this given image | | | | |(parameters/output model images will have | | | | |names like ImageName.facet.x.y, where x and y | | | | |are 0-based facet indices varying from 0 to | | | | |*nfacet-1*). The facets are merged together | | | | |into a single image in the restore solver | | | | |(i.e. it would happen only if *restore* is | | | | |true). | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.polarisation |vector |["I"] |Polarisation planes to be produced for the | | | | |image (should have at least one). Polarisation| | | | |conversion is done on-the-fly, so the output | | | | |polarisation frame may differ from that of the| | | | |dataset. An exception is thrown if there is | | | | |insufficient information to obtain the | | | | |requested polarisation (e.g. there are no | | | | |cross-pols and full stokes cube is | | | | |requested). Note, ASKAPsoft uses the *correct*| | | | |definition of stokes parameters, | | | | |i.e. *I=XX+YY*, which is different from casa | | | | |and miriad (which imply I=(XX+YY)/2).The code | | | | |parsing the value of this parameter is quite | | | | |flexible and allows many ways to define stokes| | | | |axis, e.g. ["XX YY"] or ["XX","YY"] or "XX,YY"| | | | |are all acceptable | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.nterms |int32 |1 |Number of Taylor terms for the given image. If| | | | |greater than one, a given number of Taylor | | | | |terms is generated for the given image which | | | | |are named ImageName.taylor.x, where x is the | | | | |0-based Taylor order (note, it can be combined| | | | |with faceting causing the names to be more | | | | |complex). This name substitution happens | | | | |behind the scene (as for faceting) and a | | | | |number of images (representing Taylor terms) | | | | |is created instead of a single one. This | | | | |option should be used in conjunction with | | | | |*visweights* (see above) to utilize | | | | |multi-scale multi-frequency algorithm. With | | | | |*visweights="MFS"* the code recognizes | | | | |different Taylor terms (using _taylor.x_ name | | | | |suffix) and applies the appropriate | | | | |order-dependent weight. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |*ImageName*.facetstep |int32 |min(shape(0),shape(1)) |Offset in tangent plane pixels between facet | | | | |centres (assumed the same for both | | | | |dimensions). The default value is the image | | | | |size, which means no overlap between facets | | | | |(no overlap on the shortest axis for | | | | |rectangular images). Overlap may be required | | | | |to achieve a reasonable dynamic range with | | | | |faceting (aliasing from the sources located | | | | |beyond the facet edge). The alternative way to| | | | |address the same problem is the *padding* | | | | |option of the gridder (see :doc:`gridder` for | | | | |details). | +--------------------------+----------------+-----------------------+----------------------------------------------+ Output parameters ````````````````` This section describes parameters used to select which of the many possible outputs to write to disk. Note that the restored image is always written if restore=true is specified. All parameters given in the following table have **Cimager.write* prefix**, e.g., Cimager.write.residualimage = false Please leave parameter Cimager.solver.Clean.saveintermediate = true (the default) if specifying write.psfimage=true or write.residualimage=true otherwise an exception is thrown. In addition you can specify a number of keywords to add to the header of the output images using parameters with **Cimager.header. prefix**, e.g., Cimager.header.OBSERVER = ["Joe Bloggs","name of observer"] Cimager.header.SBID = ["32385","Scheduling Block Identifier","INT"] Note that you can specify an optional comment in the second entry of the string vector and the type of the keyword ("INT","DOUBLE" or "STRING") as optional 3rd entry. The default type is string. +--------------------------+----------------+-----------------------+----------------------------------------------+ |**Parameter** |**Type** |**Default** |**Description** | +==========================+================+=======================+==============================================+ |write.residualimage |bool |false |Write the residual image(s) if true | | | | |(replaces residuals parameter) | +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.psfrawimage |bool |false |Write the 'raw' (unnormalised, | | | | |non-preconditoned) psf if true | +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.psfimage |bool |true |Write the preconditioned psf if true | +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.weightsimage |bool |false |Write the weights image(s) if true | | | | |If both weightsimage and weightslog are false | | | | |write a header keyword or table | +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.modelimage |bool |false |Write the model image(s) if true. Default is | | | | |true if restore is false or writeAtMajorCycle | | | | |is true | +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.maskimage |bool |false |Write the mask image(s) if true (cont. only)| +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.sensitivityimage |bool |false |Write the sensitivity image(s) if true. | | | | | (continuum only)| +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.firstrestore |bool |false |Write the image(s) produced in the first | | | | |restore iteration if true (continuum only)| +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.grids |bool |false |Write the complex uv grids (gridded vis, psf, | | | | |and pcf) if true (spectral imaging only)| +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.grids.uvcoord |bool |false |Label FITS output grid with UV coordinates | | | | |instead of (incorrect) RA/Dec coordinates. | | | | |UV Grids in casa format are always labeled UV.| | | | |FITS readers (wcslib) don't support UV well. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.grids.fft |bool |false |Write the FFT of the grids, real part only. | | | | |These are the dirty image, dirty psf and pcf | | | | |Coordinates will be RA/Dec. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.weightslog |bool |false |Write the weights to a log file. Alternative | | | | |to weights images if they are constant. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.beamlog |bool |true |Write the channel dependent restoring beam | | | | |sizes to a log file. If false write them to | | | | |the image as an attached table. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.multiple |bool |false |Write out all the images selected above for | | | | |multiple image parameters. By default, just | | | | |write the outputs for the first image. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |write.multiplemodels |bool |true |Write out all the model images, handy if you | | | | |do not want all the outputs, but need the | | | | |models for use in e.g., ccontsubtract | +--------------------------+----------------+-----------------------+----------------------------------------------+ |header. |vector |["","","STRING"] |Header keyword to write to the image header. | | | | |Replace with up to 8 character name | | | | |of the keyword. Second string is the keyword | | | | |description or comment (default no comment). | | | | |Third string is the optional type (INT,DOUBLE,| | | | |or STRING) with STRING the default. | +--------------------------+----------------+-----------------------+----------------------------------------------+ Traditional weighting (experimental) ```````````````````````````````````` The imager supports traditional weighting approach (i.e. weighting before gridding) in addition to preconditioning mentioned above. Both weighting schemes can be used together if desired, but the general intention is to use one or the other in the long term. The parameters configuring traditional weighting all have **Cimager.uvweight** prefix, e.g., Cimager.uvweight.robustness = 0.1. The feature itself is enabled if the list of effects passed via **Cimager.uvweight** parset parameter is not empty (by default it is, so no traditional weighting is done). In addition, the code takes some general gridder parameters such as **MaxPointingSeparation**, **padding** and **oversample** (see :doc:`gridder`) into account to ensure that the grid used to accumulate sample density, handling of oversampling and the sample rejection criterion are matching those used for imaging. Note, the weight grid itself is not oversampled but the knowledge of the oversampling factor used by the actual imaging gridder is required to fill the weight grid identically to the image grid. +--------------------------+----------------+-----------------------+----------------------------------------------+ |**Parameter** |**Type** |**Default** |**Description** | +==========================+================+=======================+==============================================+ |uvweight |vector |[] |Main parameter enabling computation and | | | | |application of traditional weights. If the | | | | |list is not empty, sample density is computed | | | | |(in a distributed fashion if applicable) and | | | | |passed to the effects (procedures) in the | | | | |order | | | | |they are listed in this parameter. The result | | | | |is used as the weight grid for all subsequent | | | | |imaging. Same weights apply to all polarisati\| | | | |on products and Taylor terms. Currently, the | | | | |following effects (weight calculators) are | | | | |supported: **ConjugatesAdderFFT**, **Robust** | | | | |and **Reciprocal**. The first one ensures | | | | |conjugate symmetry of the sample density | | | | |via a relatively brute | | | | |force approach using the FFT. This is necessa\| | | | |ry for robust weighting to work correctly with| | | | |its current implementation. The second option | | | | |(**Robust**) performs calculation of | | | | |the robust weights. And the last option, | | | | |**Reciprocal**, exists largely for experiment\| | | | |s. It simply computes an inverse for every | | | | |weight value above the threshold. All weight | | | | |values below the threshold are replaced by 0. | +--------------------------+----------------+-----------------------+----------------------------------------------+ |uvweight.robustness |float |None |The value of robustness, only used if there is| | | | |**Robust** among the list of effects given in | | | | |the **uvweight** parameter. The value is | | | | |supposed to be between -2. and 2., inclusive | +--------------------------+----------------+-----------------------+----------------------------------------------+ |uvweight.recipthreshold |float |1e-5 |Threshold for **Reciprocal** weight calculator| +--------------------------+----------------+-----------------------+----------------------------------------------+ |uvweight.perbeam |bool |false |If true, the weight is accumulated and applied| | | | |per beam. Otherwise, the beam index is ignored| +--------------------------+----------------+-----------------------+----------------------------------------------+ |uvweight.maxbeams |int |36 |This parameter is only used if **perbeam** | | | | |parameter is true and represents the maximum | | | | |number of beams which can be encountered in | | | | |the measurement set (beam indices are expected| | | | |to range from 0 to (maxbeams - 1), inclusive) | +--------------------------+----------------+-----------------------+----------------------------------------------+ |uvweight.alldatawt |bool |value of **perbeam** |If true, all samples are included to populate | | | | |the initial sample density grid. Otherwise, | | | | |only data from a representative beam and field| | | | |are accepted. This option is equivalent to | | | | |**alldatapsf** in the case of preconditioning,| | | | |and may be handy if the measurement set conta\| | | | |ins interleaved pointings or multiple beams. | +--------------------------+----------------+-----------------------+----------------------------------------------+ Example ------- .. code-block:: bash # # Input measurement set # Cimager.dataset = 10uJy_stdtest.ms # # Define the image(s) to write # Cimager.Images.Names = [image.i.10uJy_clean_stdtest] Cimager.Images.shape = [2048,2048] Cimager.Images.cellsize = [6.0arcsec, 6.0arcsec] Cimager.Images.image.i.10uJy_clean_stdtest.frequency = [1.420e9,1.420e9] Cimager.Images.image.i.10uJy_clean_stdtest.nchan = 1 Cimager.Images.image.i.10uJy_clean_stdtest.direction = [12h30m00.00, -45.00.00.00, J2000] # # Use a multiscale Clean solver # Cimager.solver = Clean Cimager.solver.Clean.algorithm = MultiScale Cimager.solver.Clean.scales = [0, 3, 10, 30] Cimager.solver.Clean.niter = 10000 Cimager.solver.Clean.gain = 0.1 Cimager.solver.Clean.tolerance = 0.1 Cimager.solver.Clean.verbose = True Cimager.threshold.minorcycle = [0.27mJy, 10%] Cimager.threshold.majorcycle = 0.3mJy Cimager.ncycles = 10 # # Restore the image at the end # Cimager.restore = True Cimager.restore.beam = [30arcsec, 30arcsec, 0deg] # # Use preconditioning for deconvolution # Cimager.preconditioner.Names = [Wiener, GaussianTaper] Cimager.preconditioner.Wiener.noisepower = 100.0 Cimager.preconditioner.GaussianTaper = [20arcsec, 20arcsec, 0deg]