linmos (Linear Mosaic Applicator)
This page provides instruction for using the linmos program. The purpose of
this software is to perform a linear mosaic of a set of images.
Running the program
It can be run with the following command, where “config.in” is a file containing
the configuration parameters described in the next section.
The linmos program is not parallel/distributed.
Parallel linmos (linmos-mpi)
There is a parallel version of linmos which will divide the mosaic by channel over the number of
ranks. This improves the run time markedly as the I/O is distributed. Furthermore this also
reduces the memory load. For continuum mosaics you can use multiple threads to speed up the run time.
You may need to set OMP_NUM_THREADS>0 to enable this and your installation needs to be compiled with OMP
enabled. Note that you could combine these options, but generally best performance for spectral line cubes
is obtained by parallelising over channels only.
There is an option to do the I/O using either collective operations or independently. For large
files and with Lustre striping factors of, e.g., 16 or 32 a factor of two in speed can be gained with
collective I/O. The use of collective I/O is only possible with fits images.
There is also an option to trim the empty edges of the images (containing NaNs) that can potentially reduce
the computation time and the size of the output files. Please refer to the configuration below on how to
enable this feature.
Configuration Parameters
The following table contains the configuration parameters to be specified in the config.in
file shown on above command line. Note that each parameter must be prefixed with “linmos.”.
For example, the weighttype parameter becomes linmos.weighttype.
Note
There is no default option for weighttype. This option must be set.
| Parameter |
Type |
Default |
Description |
|---|
| names |
vector<string> |
none |
Names of the input images. If these images start with
“image” and have associated sensitivity images, the latter
are integrated into a sensitivity image for the mosaic. |
| weights |
vector<string> |
null |
Optional parameter (required if using weight images). Names
of input images containing pixel weights. There must be one
weight image for each image, and the size must match.
Alternatively, a weightslog file can be specified for each
input image. Use the useweightslog parameter and/or .txt
extension to indicate weightslog inputs.
If weights are required, but not specified, linmos will try
to read them from the image headers.
Ignored if weighttype=FromPrimaryBeamModel or if
findmosaics=true. |
| useweightslog |
bool |
none |
Optional parameter to specify whether or not to use
weightslog files. If true, the “.txt” extension in the
weights parameter can be omitted. |
| stokesinames |
vector<string> |
null |
Optionally specify the Stokes I images to use when
using removeleakage. There must be one Stokes I image for
each input image, and the size must match.
If not specified, removeleakage tries to find the Stokes I
to use by replacing e.g., “.q.” by “.i.” in the name. |
| beams |
vector<int> |
null |
Optionally specify the beam numbers to use for mosaicking
and the removeleakage option. If specified the number of
beams must match the number of images.
By default the beam number is deduced from the image name
by looking for the beam tag (e.g., “.beam01”) in the name |
| beamangles |
vector<double> |
null |
Optionally specify the beam angles to use for mosaicking.
If specified the number of beam angles must match the number
of images.
By default the beam angle is taken from the alpha value in
the primary beam specification (see below). You only need to
specify this here if the angle is not the same for all
input images (e.g., if you are combining interleaves or
different epochs). Angle units are radians. |
| outname |
string |
none |
Name of the output image. Ignored if findmosaics=true. |
| outweight |
string |
none |
Name of output image containing pixel weights. Ignored if
findmosaics=true. |
| weighttype |
string |
none |
How to determine the pixel weights. Options:
- FromWeightImages: from weight images. Parameter
weights must be present and there must be a one-to-one
correspondence with the input images.
- FromPrimaryBeamModel: using a primary-beam
model. If beam centres are not specified (see below),
the reference pixel of each input image is used.
- Combined: uses both the weight images and the
PB model to form the pixel weight
|
| weightstate |
string |
Corrected |
The weighting state of the input images.
Options:
- Corrected: Direction-dependent beams/weights have
been divided out of input images.
- Inherent: Input images retain the natural
primary-beam weighting of the visibilities.
- Weighted: Full primary-beam-squared weighting.
|
| cutoff |
float |
0.01 |
Desired cutoff of the gain function used to form weights,
relative to the maximum gain. |
| psfref |
uint |
0 |
Which of the input images to extract restoring-beam
information from. The default behaviour is to use the
first image specified (indices start at 0). |
| outputref |
int |
-1 |
Specify which of the input images to use for the output
coordinates (i.e., projection centre). The default is to
pick a central field or use outputcentre if specified |
| outputcentre |
vector<string> |
[,,J2000,SIN] |
Optional parameter specifying output projection centre,
coordinates (default J2000) and projection (default SIN).
E.g., [12:34:56.7, -23.45.12.34] for J2000, RA/Dec in SIN
projection (Note “.” separator in dec);
[264.2deg,19.7deg,GALACTIC,MOL] for Galactic
coordinates in Mollweide projection.
See Projections for the list of supported projections.
See Directions for the list of supported Direction types.
For all-sky projections, like MOL or HPX, with invalid pixel
areas best results are obtained with regrid.decimate=1 |
| nterms |
uint |
-1 |
Process multiple taylor-term images. The string “taylor.0”
must be present in both input and output image names
(including weights images), and it will be incremented from
0 to nterms-1. Ignored if findmosaics=true. |
| findmosaics |
bool |
false |
Instead of specifying specific input and output files to
mosaic, search the current directory for suitable mosaics.
Parameter names is used to specify a vector of tags, and
all groups of images that have names that are equal apart
from these tags are mosaicked together. Groups must have one
image per tag. Currently only groups with prefixes of
“image” and “residual” are allowed, with prefixes “weights”
and “sensitivity” special cases that are searched for once
groups are identified. Parameters weights, outname,
outweight and nterms are ignored if findmosaic=true. |
| imagetype |
string |
“casa” |
Type of the image handler (determines the format of
the images, both which are written to or read from
the disk). The default is to create casa images but
“fits” can also be chosen. |
| imagealloc |
string |
“fast” |
Set to “fast” to use the fast file allocation scheme.
This can save a lot of time when creating large cubes.
This is now the default, set to “” to turn off |
| imageaccess |
string |
“individual” |
The imageaccess parameters are for fits images only:
Choose between “individual” and “collective” I/O operations.
For collective I/O the number of ranks must divide evenly
into the number of channels. |
| imageaccess.axis |
uint |
0 |
Specify the channel axis of the cubes - usually 3
This is only used for collective I/O on fits cubes |
| imageaccess.order |
string |
“distributed” |
How do the ranks read/write channels:
- “distributed” over the spectrum, one section per rank
- “contiguous” in rank order, a section at a time
For collective I/O “contiguous” is automatically selected
|
| imageaccess.write |
string |
“serial” |
How does the output cube get written:
- “serial” one rank at a time
- “parallel” all ranks at the same time
For casa images “serial” is used regardless.
|
| imageHistory |
vector<string> |
none |
Specify lines to add to the image history |
| calcstats |
bool |
false |
If true, calculate the image statistics and write it to the
image table. Note: only available for linmos-mpi task. |
| keywordsToCopy |
vector<string> |
<several> |
Specify the header keywords to copy from the reference image
to the output. By default TELESCOP,PROJECT,SBID,DATE-OBS and
DURATION are copied. This will override that list. |
| trimming |
bool |
false |
If true, trim the parts of the image below the beam cutoff.
This can speed up computation and reduce file size. This
feature is only supported for weighttype = Combined or
FromPrimaryBeamModel. It is only available to linmos-mpi |
If input images need to be regridded, the following ImageRegrid options are available:
| Parameter |
Type |
Default |
Description |
|---|
| regrid.method |
string |
linear |
ImageRegrid interpolation method: nearest, linear,
cubic or lanczos. |
| regrid.decimate |
uint |
3 |
ImageRegrid decimation factor. In the range 3-10 is likely
to provide the best performance/accuracy tradeoff |
| regrid.replicate |
bool |
false |
ImageRegrid replicate option. |
| regrid.force |
bool |
false |
ImageRegrid force option. |
Definition of beam centres
If weights are generated from primary-beam models (weighttype=FromPrimaryBeamModel), it is possible to set the
beam centres from within the parset. Since this is most likely useful when each input image comes from a different
multi-beam feed, feeds offset parameters from other applications are used for this. If the origin of the beams
offset system is not specified, using either feeds.centre or feeds.centreref, any offsets are ignored and the
reference pixel of each input image is used as the primary-beam centre.
The feeds parameters can be given either in the main linmos parset or a separate offsets parset file set by the
feeds.offsetsfile parameter.
| Parameter |
Type |
Default |
Description |
|---|
| feeds.centre |
vector<string> |
none |
Optional parameter (it or feeds.centreref required when
specifying beam offsets).
Two-element vector containing the right ascension and
declination that all of the offsets are relative to. |
| feeds.centreref |
int |
none |
Optional parameter (it or feeds.centre required when
specifying beam offsets). Which of the input images to use
to automatically set feeds.centre. Indices start at 0.
If neither of these parameters are set, the reference pixel
of each input image is used as the primary-beam centre. |
| feeds.spacing |
string |
none |
Optional parameter (required when specifying beam offsets
in the main linmos parset). Beam/feed spacing when giving
offsets in the main linmos parset. If feeds.offsetsfile
is given, this parameter will be ignored. |
| feeds.names[i]
(one per input
image) |
vector<string> |
none |
Optional parameter (required when specifying beam offsets
in the main linmos parset). Two-element vector containing
the beam offset relative to the feeds.centre parameter.
Offsets correspond to hour angle and declination.
names[i] should match the names of the input images,
given in linmos.names (see above). If feeds.offsetsfile
is given, these parameters will be ignored. |
| feeds.offsetsfile |
string |
none |
Optional parameter. Name of the optional beam/feed offsets
parset. If present, any offsets specified in the main
linmos parset will be ignored. |
| feeds.names |
vector<string> |
none |
Optional parameter (required either here or below when
specifying a beam offsets parset). The beam offsets parset
should have one line per input image, with parameter keys
(minus the feeds. prefix) specified by this parameter. If
the offsets parset also contains a names parameter, the
main linmos entry will hold, to allow a subset of beams
from a general to be chosen. |
If feed offsets are provided via an additional parset (i.e. not that one passed directly to
the linmos program), the file shall have the following format:
Note
These parameters, specified in an external file, do not require the “limos.” prefix.
| Parameter |
Type |
Default |
Description |
|---|
| feeds.names |
vector<string> |
null |
Optional parameter (required either here or above when
specifying a beam offsets parset). The beam offsets parset
should have one line per input image, with parameter keys
(minus the feeds. prefix) specified by this parameter. If
the offsets parset also contains a names parameter, the
main linmos entry will hold, to allow a subset of beams
from a general to be chosen. |
| feeds.spacing |
string |
none |
Beam/feed spacing. When using this extra offsets parset,
the spacing needs to be specified in this parset. |
| feeds.beamnames[i]
(one per input
image) |
vector<string> |
none |
Two-element vector containing the beam offset relative to
the feeds.centre parameter. Offsets correspond to hour
angle and declination. beamnames[i] should match the
names given in feeds.names* (see above). |
Alternate Primary Beam Models
It is possible to select the model that is used for the weighting. This is selected in the linmos parset by
the key “primarybeam”
| Parameter |
Type |
Default |
Description |
|---|
| primarybeam |
string |
“GaussianPB” |
Optional parameter that allows the user to select which
primary beam will be used in weighting. The parameters of
which can also be altered if required. Also supported are
ASKAP_PB, MWA_PB and SKA_PB. |
Gaussian Primary Beam Options
You can choose the aperture size and scaling parameters both of the FWHM of the beam and a scaling of the exponent.
In the parfile these are sub parameters of the Primary beam type. (e.g linmos.primarybeam.GaussianPB.aperture)
The default Gaussian Primary beam is now 2 dimensional. But unless the user specifies x and w widths they just get the symmetric beam as defined by the aperture.
| Parameter |
Type |
Default |
Description |
|---|
| aperture |
double |
12 |
Aperture size in metres. |
| fwhmscaling |
double |
1.09 |
Scaling of the full width half max of the Gaussian |
| expscaling |
double |
4 log(2) |
Scaling of the primary beam exponent |
The 2 dimensional beam is governed by the following parameters.
| 2D-Parameters |
Type |
Default |
Description |
|---|
| (x/y)width |
double |
0.0 |
Angular width in rad. of the x (N-S) and y (E-W) Gaussian |
| (x/y)off |
double |
0.0 |
Angular offset from nominal beamcentre in rad., E, N are +ve |
| alpha |
double |
0.0 |
PA in rad. measured from North in an +ve RA direction |
ASKAP Primary Beam Options
The ASKAP primary beam is specified using a 4D or 5D image file determined from antenna beam measurements
(holography).
There are two options to specify the frequency dependence of the beam:
- Taylor term planes (nterms > 0) or,
- a frequency cube (nterms = 0)
Use the image parameter to specify the beam file (FITS format, axes RA, Dec, Taylor term or Frequency, optionally
polarisation, and beam number).
The image file can contain more Taylor planes, but only the first nterms planes (up to 3) are used.
In the parfile these are sub parameters of the Primary beam type. (e.g linmos.primarybeam.ASKAP_PB.nterms).
A beam file with 4 axes can only be used to correct the primary beam response. A 5D file with a polarisation axis
can be used to correct both primary beam response and leakage.
| Parameter |
Type |
Default |
Description |
|---|
| image |
string |
none |
FITS image file with 4/5 axes (see above), note that you
need to specify the full filename (including .fits) here |
| nterms |
integer |
0 |
Number of Beam Taylor terms to use, or 0 for a frequency
cube |
| alpha |
double |
0 |
Position angle of the beam, optional rotation of measured
beam pattern. |
| xscale,yscale |
double |
from FITS |
Override x and y axis increments found in FITS header |
| freqoffset |
double |
from FITS |
Override reference frequency found in FITS header |
| freqscale |
double |
from FITS |
Override frequency increment found in FITS header |
| interpolation |
string |
linear |
Interpolation method: nearest, linear, cubic or
lanczos. |
MWA Primary Beam Options
| Parameter |
Type |
Default |
Description |
|---|
| latitude |
double |
-26.703319 deg |
Array latitude in radians |
| longitude |
double |
116.67081 deg |
Array longitude in radians |
| dipole.separation |
double |
1.10 metres |
Dipole separation |
| dipole.height |
double |
0.30 metres |
dipole hheight |
Primary Beam Corrections to the Taylor terms
The primary beam is a function of frequency. Therefore the apparent spectral index of a point source away from beam centre
will contain a contribution from the frequency dependence of the primary beam. It is possible to estimate this contribution
and remove it by scaling the Taylor term images appropriately.
This is an analytic correction for Gaussian beams.
For ASKAP_PB it is calculated either by numerical differentiation (for spectral beam cubes) or
by evaluation of the beam Taylor terms.
| Parameter |
Type |
Default |
Description |
|---|
| removebeam |
bool |
false |
Remove beam from the Taylor term images |
Leakage Corrections
The ASKAP holography beam measurements also measure the off-axis leakage across the field. We can use these
measurements to correct the Stokes Q, U and V terms for leakage from Stokes I. You will need to specify
a 5D FITS image file with axes Ra, Dec, Freq, polarisation and beam containing the holography measurements.
Example 5 below shows how to specify this.
Note that the beam images will be interpolated in the spatial directions, but for frequency the nearest plane will be
used and scaled to the correct frequency.
The stokesinames and beams parameters can be used to specify which Stokes I image and beam number to use for each
input image if the naming scheme is non standard (i.e., does not contain polarisation and beam tags like “.q.” and
“.beam04”)
| Parameter |
Type |
Default |
Description |
|---|
| removeleakage |
bool |
false |
Remove off-axis leakage from Stokes Q, U and V images using
a model of the Stokes-I dependent leakage |
Examples
Example 1:
Example linmos parset to combine individual feed images from a 36-feed simulation. Weights
images are used to weight the pixels.
linmos.weighttype = FromWeightImages
linmos.names = [image_feed00..35_offset.i.dirty.restored]
linmos.weights = [weights_feed00..35_offset.i.dirty]
linmos.outname = image_mosaic.i.dirty.restored
linmos.outweight = weights_mosaic.i.dirty
Example 2:
Example linmos parset to combine the four inner-most feed images from a 36-feed observation.
Gaussian primary-beam models are used to weight the pixels. The primary-beam offsets are
provided in an external file.
linmos.weighttype = FromPrimaryBeamModel
linmos.names = [image_feed14..15.i.dirty.restored, image_feed20..21.i.dirty.restored]
linmos.outname = image_mosaic.i.dirty.restored
linmos.outweight = weights_mosaic.i.dirty
linmos.feeds.centre = [12h30m00.00, -45.00.00.00]
# specify a beam offsets file
linmos.feeds.offsetsfile = linmos_beam_offsets.in
# Specify which feeds from the "offsetsfile" (specified above) are to be used
linmos.feeds.names = [PAF36.feed14..15, PAF36.feed20..21]
Below is the linmos_beam_offsets.in file refered to in the above parameter set:
feeds.spacing = 1deg
<snip>
feeds.PAF36.feed14 = [-0.5, -0.5]
feeds.PAF36.feed15 = [-0.5, 0.5]
<snip>
feeds.PAF36.feed20 = [0.5, -0.5]
feeds.PAF36.feed21 = [0.5, 0.5]
<snip>
Example 3:
Example linmos parset to combine the four inner-most feed images from a 36-feed simulation.
The primary-beam offsets directly in the parameter set.
linmos.weighttype = FromPrimaryBeamModel
linmos.names = [image_feed14..15.i.dirty.restored, image_feed20..21.i.dirty.restored]
linmos.outname = image_mosaic.i.dirty.restored
linmos.outweight = weights_mosaic.i.dirty
linmos.feeds.centre = [12h30m00.00, -45.00.00.00]
linmos.feeds.spacing = 1deg
linmos.feeds.image_feed14.i.dirty.restored = [-0.5, -0.5]
linmos.feeds.image_feed15.i.dirty.restored = [-0.5, 0.5]
linmos.feeds.image_feed20.i.dirty.restored = [0.5, -0.5]
linmos.feeds.image_feed21.i.dirty.restored = [0.5, 0.5]
Example 4:
Example linmos-mpi parset to combine individual feed images from a 36-feed simulation for each of three
separate taylor terms 0, 1 and 2. The location of taylor.* in all inputs and outputs is given explicitly.
Uses combined weighting, which includes primary beam response and weight images, and removebeam to
remove the spectral index of the beam from the results.
Uses the ASKAP_PB with 3 terms from a Taylor term beam FITS image derived from holography measurements.
linmos.weighttype = Combined
linmos.weightstate = Inherent
linmos.names = [image_feed00..35_offset.i.dirty.taylor.0.restored]
linmos.weights = [weights_feed00..35_offset.i.dirty.taylor.0]
linmos.outname = image_mosaic.i.dirty.taylor.0.restored
linmos.outweight = weights_mosaic.i.dirty.taylor.0
linmos.nterms = 3
linmos.removebeam = true
linmos.primarybeam = ASKAP_PB
linmos.primarybeam.ASKAP_PB.image = askap-beam-taylor.fits
linmos.primarybeam.ASKAP_PB.nterms = 3
Example 5:
Example linmos-mpi parset to combine polarisation Q image cubes for 36 beams while removing off axis leakage.
Uses combined weighting, which includes primary beam response and weight images.
Uses the ASKAP_PB with a 5D FITS image derived from holography measurements containing
the primary beam response for Stokes I and the Stokes I leakage for Q, U and V for each channel
and position.
This example also uses a decimate value of 10 for regridding and collective I/O distributed over the channel
axis to try and speed up processing.
linmos.weighttype = Combined
linmos.weightstate = Inherent
linmos.imagetype = fits
linmos.names = [image.restored.q.SB10007.contcube.POSSUM_2126-54.beam00..35.conv]
linmos.weights = [weights.q.SB10007.contcube.POSSUM_2126-54.beam00..35]
linmos.outname = image.restored.q.SB10007.contcube.POSSUM_2126-54.linmos.conv
linmos.outweight = weights.q.SB10007.contcube.POSSUM_2126-54.linmos.convrestored
linmos.primarybeam = ASKAP_PB
linmos.primarybeam.ASKAP_PB.image = askap-beam-cube_16881.fits
linmos.removeleakage = true
linmos.imageaccess = collective
linmos.imageaccess.axis = 3
linmos.regrid.decimate = 10
Example 6:
Example linmos parset to combine individual feed images from a 36-feed simulation. A mosaics is made for each set
of 36 images that has one image for each tag (param “names”) but filenames that are otherwise the same. Only the
“image” and “residual” prefixes are currently supported. For example, if the outputs produced for Data Challenge 1A
were produced for each feed and stored in a single directory, the following mosaics would be made:
image_linmos.i.clean.taylor.0, image_linmos.i.clean.taylor.0.restored, image_linmos.i.clean.taylor.1,
image_linmos.i.clean.taylor.1.restored, image_linmos.i.dirty.restored, residual_linmos.i.clean.taylor.0 and
residual_linmos.i.clean.taylor.1. Associated weights and sensitivity images would also be made, however in
situations where multiple mosaics have the same weights or sensitivites (e.g. image_linmos.i.clean.taylor.0,
image_linmos.i.clean.taylor.0.restored and residual_linmos.i.clean.taylor.0), only one would be made.
Furthermore, since the DC1A does not seem to produce weights.*.taylor.2 and we have specified weighttype
FromWeightImages, mosaic image_linmos.clean.taylor.2 would not be made. It would be produced if weighttype were
FromPrimaryBeamModel.
linmos.weighttype = FromWeightImages
linmos.findmosaics = true
linmos.names = [feed00..35_offset]