Installing and Updating Miriad

Installing and updating Miriad

Miriad , the radio-astronomy data reduction package used by the ATNF, has been ported to a variety of unix-based platforms. At the ATNF it runs under Linux (64-bit); SunOS 5.9 (aka Solaris 2.9) and later; and Intel Macs running MacOSX 10.6.2 ("Snow Leopard") or later which is based on Darwin 10.2.0. Binary distribution kits are provided for these platforms.

In the past (and present) Miriad has also run on 32-bit Linux, PowerPC Macs, in the Windows Cygwin environment, on SGI/IRIX (32- and 64-bit), OSF/1 (aka Digital UNIX or Tru64), HP/HPUX, Cray/UNICOS, and the ConvexOS, and SunOS 4.x operating systems, (and it also originally ran on VAX/VMS). Although these platforms are not directly supported by ATNF a GNU-autoconf-derived build system is available for them and other unix-based platforms.

Contents

 
miriad@atnf.csiro.au


Instructions for installing Miriad

For a quick binary install guide to get Miriad on your Mac, look here, otherwise, read on.

Miriad is distributed via anonymous ftp as bzip2'd tar files which are updated weekly. For convenience the Miriad homepage contains hypertext versions of the User and Programmer Guides, and these are also available as separate bzip2'd PostScript files (both forms are included in the distribution).

Required third-party software

Miriad relies on the following third-party software and libraries:

Of these, most Linux distributions will contain packages for all but RPFITS and WCSLIB. You will probably have to install RPFITS from source code unless you can use one of the binary distribution kits. Installation of WCSLIB from its source code is straightforward, (configure ; gmake), or you may be able to find a pre-built package. If you are building from source on a platform for which a binary Miriad distribution is available, a quick way to resolve the above dependencies is to install the binary distribution before running configure.

Customising array sizes

One advantage of installing from source code is that it provides an opportunity to adjust the following parameters in the Fortran include files $MIR/inc/maxdim.h and tmpdim.h to reflect your problem sizes and memory availability:

	MAXBUF  – Maximum general scratch buffer size.
	MAXDIM  – Maximum number of pixels on an image axis.
	MAXCHAN – Maximum number of spectral channels.
	MAXANT  – Maximum number of antennas.
	MAXBASE – Maximum number of baselines (normally derived from
                        MAXANT).
	MAXWIN  – Maximum number of simultaneous frequency bands
                        ("windows").
	MAXWIDE – Maximum number of wideband channels (Hat Ck specific).
and, in $MIR/inc/maxnax.h,
	MAXNAX  – Maximum number of image axes.
Except for MAXBUF, these parameters must also be set for C source code in $MIR/inc/maxdimc.h. Please read the comments in the prologue of that file.

No Miriad code allocates memory for arrays of length more than a few times any of the parameters in maxdim.h (e.g. there are no arrays of size MAXDIM**2). Miriad programs mostly allocate memory dynamically when needed. tmpdim.h is similar to maxdim.h except the values are invariably set lower because tasks that use it do statically declare arrays of size MAXDIM**2 or similar.

Include files may exist within architecture-specific subdirectories of $MIR/inc and take preference if so; this search-path mechanism allows multiple architectures to be maintained with a single set of source code. For example, since 64-bit systems typically have more memory, $MIR/inc/linux64/maxdim.h defines larger values of MAXBUF and MAXDIM for the linux64 architecture and is the relevant one to change on those systems.

Installing from a binary distribution

If a binary distribution kit is available for your platform it is worth trying to install it first. However, even though the PGPLOT, RPFITS, and WCSLIB libraries are statically linked there is still a chance that the executables may not run because of incompatibilities between versions of dynamic (aka shared) system or compiler libraries. Nevertheless PGPLOT, RPFITS, and WCSLIB are supplied and this may save you from having to install them yourself.

Note that if you don't have an account at the ATNF that would allow you to run mirsync and you intend to keep your installation up-to-date by running mirimport then you will have to install Miriad from source code.

  1. Download a copy of the Miriad distribution from ftp://ftp.atnf.csiro.au/pub/software/miriad using your web browser or wget, or by running ftp from the command line.

    You will need the common runtime files, miriad-common.tar.bz2 (which includes catalogues, auxiliary data files and also the compiled documentation) and the binaries, which will be in a bzip2'd tar file with a name like miriad-linux64.tar.bz2 (others currently are miriad-darwin_x86_64.tar.bz2, miriad-sun4sol.tar.bz2).

    If you are running an older version of linux you may get error messages like: "uvplt: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by uvplt)". This indicates your kernel version is older than what is needed. You can give this version a try: miriad-linux64.old.tar.bz2, or build from source.

    Note: all recent Macs should be able to execute the x86_64 instruction set. If the arch command reports i386 then ignore that and instead check that the MACHTYPE environment variable is set to x86_64.

  2. The distribution kits unpack into subdirectory ./miriad so if you wanted the Miriad root directory to be /usr/local/miriad, then you might use something like

        % cd /usr/local
        % bzcat miriad-common.tar.bz2 | tar xvf -
        % bzcat miriad-linux64.tar.bz2  | tar xvf -
    

    Multiple architectures can safely be downloaded and live alongside one another without conflict. Allow about 100 MiB for each binary format.

  3. Install the Miriad initialization scripts. We use the environment variable MIR to indicate the root directory. Create this, e.g.

        % MIR=/usr/local/miriad; export MIR
    
    in
    sh-like shells (e.g. bash, ksh, ash, zsh), or
        % setenv MIR /usr/local/miriad
    
    in csh-like shells (e.g. tcsh), then use
        % cd $MIR
        % sed -e "s,@MIRROOT@,$MIR," scripts/MIRRC.in > MIRRC
        % sed -e "s,@MIRROOT@,$MIR," scripts/MIRRC.sh.in > MIRRC.sh
        % chmod 644 MIRRC*
    
    This does a textual replacement of the string "@MIRROOT@" in the template file $MIR/scripts/MIRRC.in with the value of $MIR and puts the result in MIRRC. Likewise for MIRRC.sh.

Refer to the post-installation instructions for initialization and testing.

Installing from source code

Source code management for ATNF Miriad was completely revised in late 2006. The old Install script was replaced with a system based on GNU autoconf and the functions performed by mircadd, mirclnk, mirfind, mirlibs, mirnewer, and mirupdate, and related utilities were subsumed by GNU make. The new build system implements full dependency analysis, both of sources on included files, and of executables on static library modules.

Most Miriad code is written in fairly standard Fortran 77 and C. However, both a Fortran and C preprocessor are used to add some simple language extensions and to aid portability. These preprocessors are compiled automatically before any of the Miriad source code.

The Fortran preprocessor, ratty, must be applied to all files with a .for suffix. It makes VAX-Fortran extensions universally available, and provides support for conditional compilation directives (little used) and machine-independent compiler optimisation directives. ratty output generally differs very little from the input and looks like fairly vanilla Fortran (with a few VAX extensions).

The C preprocessor, intf2c, (which is additional to the normal C preprocessor) must be applied to all .f2c files. It's only purpose is to aid in calling C functions from Fortran; passing standard C through it simply prepends a few lines. A special syntax is used for function declarations and their dummy arguments when these functions are to be called by Fortran.

See the preamble comments of ratty.c and intf2c.c (in the tools subdirectory) for more information. It is beyond the scope of this note to explain how ratty and intf2c might need to be adapted to support a new architecture.

To build Miriad from source code you will need

Ideally, once the required compilers, utilities and third-party libraries have been installed, building Miriad from source code should be as simple as

    % configure
    % gmake
However, in practice there are usually a few complications and some manual tweaking is likely to be required.

  1. Before anything else you must install the compilers, utilities and third-party libraries listed above. Most Linux systems provide packages for all but RPFITS and possibly WCSLIB which you may need to install from source code.

  2. Download a copy of the Miriad source code distribution from ftp://ftp.atnf.csiro.au/pub/software/miriad using your web browser, wget, or by running ftp from the command line.

    You will need the source code, miriad-code.tar.bz2, and common runtime files, miriad-common.tar.bz2, which includes catalogues, auxiliary data files and also the compiled documentation. (If you're interested, the RCS version files are also available in miriad-rcs.tar.bz2, these will show exactly what changes have been made to the source files in recent times.)

  3. Like most software distribution kits, Miriad unpacks into a separate subdirectory, ./miriad. Thus, if you wanted the Miriad root directory to be /usr/local/miriad you might use something like

        % cd /usr/local
        % bzcat miriad-code.tar.bz2   | tar xvf -
        % bzcat miriad-common.tar.bz2 | tar xvf -
    

    It is possible to build and maintain multiple architectures alongside one another without conflict. Allow about 100MiB for each.

  4. Before running configure check that your architecture is recognized by Miriad:

        % cd /usr/local/miriad/scripts
        % ./mirarch
    
    If mirarch reports "unknown" you can either leave it at that or attempt to provide a more sensible value by appropriate editing of mirarch (a simple shell script) – in the next step configure will use it to determine the architecture. Note that you will not be able to run multiple architectures if more than one is unknown.

    From now on we will refer to the Miriad root directory as $MIR and the architecture reported by mirarch as $MIRARCH. You may find it convenient to create these as environment variables.

  5. Now run configure

        % cd $MIR
        % ./configure
    
    If you want to help configure find particular versions of the Fortran or C compilers you can set the F77 and/or CC environment variables before running it (however, note that the other options listed by 'configure --help' are ignored).

  6. If you're lucky, this step will involve no work other than checking. Otherwise it may require anything from re-defining a few variables to installing additional software.

    Peruse the output generated by configure and address any errors or warnings, particularly relating to third-party libraries. If necessary, install compilers or libraries that were not found (if they are already installed, just not found, read on). Try to resolve as many problems as possible before proceeding to the next step. You can re-run configure as many times as required.

    configure produces three files, these are its only effect. The first two, MIRRC and MIRRC.sh, reside in $MIR and are constructed by simple textual replacement of the value of the Miriad root directory from the corresponding template files (MIRRC.in, etc.) that reside in $MIR/scripts.

    The third and by far the most important file generated by configure is $MIR/$MIRARCH/GNUmakedefs (referred to as the local GNUmakedefs). This is where all of the definitions peculiar to your system should be made. You will notice that there are other GNUmakedefs files: $MIR/GNUmakedefs (the generic GNUmakedefs) contains generic variable definitions and also defines rules common to all Miriad makefiles. Also, if present, $MIR/scripts/$MIRARCH/GNUmakedefs may make a few specialized overriding definitions for recognized platforms. Every Miriad GNUmakefile reads the generic GNUmakedefs, this in turn reads the specialized GNUmakedefs (if it exists) and then the local GNUmakedefs; each can override variable definitions made previously. However, after including the specialized and local files, the generic GNUmakedefs defines further variables that are of a fundamental nature and cannot be altered. Only variables demarked by "START OF DEFAULT DEFINITIONS" and "END OF DEFAULT DEFINITIONS" in the generic GNUmakedefs can be modified in your local GNUmakedefs.

    If any unresolved problems remain from running configure then fix them by editing your local GNUmakedefs appropriately. For example, if you have a library installed in an odd location that configure did not guess, then fix it here. The meaning of each variable is explained in the generic GNUmakedefs (which must not be modified, read its prologue). To see what gmake thinks the final variable definitions are you can use

        % gmake show
    
    from any Miriad directory that contains a GNUmakefile. Of the many reported, only those variables relating to compilers and libraries should be of any consequence and they are the only ones that can be modified. You may also find
        % gmake help
    
    helpful.

    Check that $MIR/inc/rpfits.inc, delivered with the Miriad source code, matches the corresponding file that was (or should have been) installed with the RPFITS library. If not, it probably means that your version of librpfits.a is stale and you will need to update it (this should never be a problem if it came from a recent Miriad binary distribution).

    Likewise, the version reported at the top of $MIR/inc/wcslib/CHANGES should match that of the version of libwcs.a that you have installed (as it should always do if it came from a recent Miriad binary distribution).

    If you are porting Miriad to an unrecognized platform, scan the generic GNUmakedefs for the string "$(MIRARCH)", this will give you a pretty good idea of what is required. There is usually very little to be done, the main distinction is between big- or little-endian IEEE floating point formats which is controlled from GNUmakedefs via the computed variable SUBS$(MIRARCH), as explained in GNUmakedefs. The preprocessors, ratty, intf2c, and cpp, can often be appeased simply by substituting $(MIRARCH) with the name of a similar system as is done in several of the specialized GNUmakedefs which you can use for guidance. In the worst case you may need to modify Miriad code. If so, please report the changes required so that others may benefit from them.

    Once edited your local GNUmakedefs becomes a precious file that you should back-up. If you should happen to re-run configure and it finds a pre-existing GNUmakedefs it will preserve it as GNUmakedefs.bak (each time) which provides one chance to save it. Preferably you should give your backup copy another name.

  7. To start the build simply type

        % cd $MIR
        % gmake
    
    Compilations are verbose enough to allow you to see the more important commands such as compiler invokations. Problems, if any, will most likely arise when the tasks are linked and usually consist of reports of missing symbols, sometimes with inscrutible names. If necessary, and if you can, identify the libraries that contain the missing symbols, go back to
    step 6 and revise GNUmakedefs. You can then safely restart the build and it should compile only what failed the previous time.

    You can invoke gmake in any Miriad directory that contains a GNUmakefile, it will recursively rebuild that directory and all of its subdirectories. If for some reason you need to force a rebuild from scratch it should be enough to delete the library

        % rm -f $MIR/$MIRARCH/lib/libmir.a
    
    If you can't locate the source of missing symbols, or encounter some other insoluble problem, refer to trouble-shooting below.

Refer to the post-installation instructions for initialization and testing.


Post-installation

A few more steps are required to finalize the installation:

  1. Define your Miriad environment by invoking the initialization scripts

        % . MIRRC.sh
    
    in sh-like shells, or
        % source MIRRC
    
    in csh-like shells. Among others this will define an environment variable called MIRBIN which you should add to your PATH environment variable. Add these commands to your .login (for csh) and/or .profile (for sh) files to have the environment set up whenever you log in.

  2. Test the installation by running a few tasks from the miriad shell, for example atlod which will test the RPFITS library, and uvplt which will test PGPLOT, including X11.

    If there are problems refer to trouble-shooting below.

    For binary installations, if there are problems that you can't resolve, particularly if the dynamic linker complains about missing symbols, then you may have to resort to installing from source code.

    For source-code installations, if you have problems that can't be resolved by repairing GNUmakedefs then you may seek assistance from miriad@atnf.csiro.au.

  3. Finally, check out the Data Reduction area on the ATCA forum . You can sign up to post questions on the forum and receive notifications of new posts.


Trouble-shooting and system-dependent issues

Miriad is known to have been installed on a number of other platforms, either at the ATNF or elsewhere, but they are no longer directly supported by ATNF. We have minimal information on their current status but would be pleased to add notes provided from elsewhere.


Getting regular updates

Those with ssh access to an ATNF computer account might consider using rsync to update their Miriad installation. Only the relevant subdirectories need be updated from venice.atnf.csiro.au:/nfs/atapplic/miriad/. A script, $MIR/scripts/mirsync, is available that will refresh any files or subdirectories in $MIR that are present on the master and delete anything it finds within those subdirectories that is not present on the master. Use the -d (dry-run) option to mirsync to see what it would do without actually changing anything.

Most people will need to use the -u (username) option to mirsync, and supply their ATNF user name as the argument. If this option is not specified, it will attempt to login to venice with the user name of the account running mirsync, which will almost certainly be different to one's ATNF user name.

The full Miriad distribution tar files are regenerated weekly and are normally ready for collection some time before midnight on Thursday Sydney time (UTC+1000 in non-daylight-savings months). It would not be difficult to write a script to fetch and install these automatically.

Incremental source code updates are generated nightly and are usually ready for collection shortly after 2300 Sydney time. If you have installed Miriad from source code you can keep your system up-to-date by running

  % mirimport
This uses ftp to fetch the updates from the ATNF, then unpacks and installs them. You must then invoke gmake from within $MIR to rebuild Miriad.

You can set this up to run periodically as a cron job which must

For example, the crontab entry might look like
  30 22 * * 4 . $HOME/.profile ; mirimport && gmake -C $MIR
where it is assumed that the Miriad environment and a sensible PATH are defined in $HOME/.profile.


The Miriad directory structure

The Miriad runtime system consists of a number of directories that are common to all architectures (distributed as miriad-common.tar.bz2) and some that are platform-specific. The latter exist as subdirectories of $MIR/$MIRARCH (otherwise referred to as $MIRARCHD), where $MIRARCH is the Miriad platform name, e.g. linux64, darwin_x86_64, etc.

$MIR Miriad root directory, contains MIRRC, PostScript User and Programmer Guides, etc.
/$MIRARCH Platform-specific area(s) with names like linux64, darwin_x86_64, etc., contains local GNUmakedefs.
/aux Files associated with the dependency analysis.
/bin Executables.
/lib Object libraries.
/man Unix manual pages. The man1 subdirectory may be set up as a symlink to $MIR/man/man1/.
/tmp Temporary area used when compiling.
/cat Source and emission line catalogues, antenna location files, observatory coordinate database, etc.
/doc doc files (online help) for each task.
/html doc files, User and Programmer Guides converted to HTML form.
/man doc files converted to Unix man pages.
/scripts Utility scripts, the most important being mirarch. Platform-specific subdirectories contain specialized GNUmakedefs.
/specdoc "Special" doc files, not associated with a particular task (cheat, help, miriad, tasks, etc.).

Miriad source code and utilities (distributed as miriad-code.tar.bz2) exist in the following directories:
$MIR Miriad root directory, contains generic GNUmakedefs and top-level GNUmakefile.
/guides LaTeX source and several utilities for compiling the User and Programmer Guides.
/inc Commonly-used Fortran and C include files.
/linpack LINPACK source code.
/prog Fortran source code for Miriad tasks.
/spec/* Source code for special-case utilities, the main ones being the xmtv and xpanel.
/subs Source code, mainly Fortran but some C, for the Miriad library. Platform-specific variants of a few source files are contained in subdirectories.
/tools C source code for Miriad tools, including the miriad shell, intf2c and ratty preprocessors, and utilities for extracting documentation from source code and reformatting it.


Software development in Miriad

Miriad's GNUmakefiles have a dual function; programmers may develop Miriad software in a private workspace using the same GNUmakefiles that are used to build the Miriad system. The GNUmakefiles themselves detect the difference by the context in which they are invoked. They contain different sets of rules for each purpose, and also some rules that are common to both. Programmer builds, like system builds, implement full dependency analysis, both of sources on included files, and of executables on static library modules.

This dichotomy extends to GNUmakedefs which contains two separate sets of compiler options. The first set, FCOPT, CPPOPT, COPT and LDOPT, nominally defined with optimization flags, is intended for system compilation. The second set, FCDBG, CPPDBG, CDBG and LDDBG, nominally with debug compiler flags, is intended for programmer compilations.

To create a Miriad workspace, first ensure that the Miriad environment is defined (e.g. source MIRRC), then

  cd ~
  mirtree
This will create a subdirectory with a name matching $MIR (usually miriad) which in turn contains subdirectories that mirror the Miriad directory hierarchy. If your Miriad installation contains a copy of the RCS repository (unlikely) then each subdirectory should contain a symlink called RCS that points to the appropriate subdirectory of the repository. Otherwise, each should contain a symlink to the relevant GNUmakefile.

To develop code for multiple architectures, run mirtree once on a machine of each architecture that is to be supported.

A private debug library may be created as follows:

  cd ~/miriad/subs
  gmake lib
The library and preprocessed Fortran files (the source files for the debugger) are deposited in ~/miriad/$MIRARCH/lib, where $MIRARCH is the Miriad architecture. It is good practice to maintain a full debug version of libmir.a and ensure that it is up-to-date before attempting to debug a task.

Compiling a Miriad task, say atlod, is as simple as

  cd ~/miriad/prog
  cp $MIRPROG/atlod.for .
  gmake
or simpler yet if RCS repositories are available:
  gmake -C ~/miriad/prog atlod
The executable will be linked preferentially to your own debug library (if it exists), with any remaining unresolved symbols satisfied from the Miriad system library, $MIRLIB/libmir.a. The debug version of the executable is deposited in ~/miriad/$MIRARCH/bin. You may find it convenient to add this directory to PATH.

To see a full list of GNUmakefile targets (different for each directory):

  cd ~/miriad/subs              ...for example
  gmake help

It is possible to control GNUmakefile variables by making private definitions in ~/miriad/$MIRARCH/GNUmakedefs. For your workspace alone, these override the settings in $MIR/GNUmakedefs and $MIR/$MIRARCH/GNUmakedefs – particularly useful for experimenting with compiler options. To get a context-sensitive list of GNUmakefile variables with personal overrides (different for each directory):

  gmake show

Finally, it is good practice to clean up private executables when you've finished with them otherwise you might forget later that you're getting a private version:

  cd ~/miriad/prog
  gmake cleanest


$Id: INSTALL.html,v 1.32 2017/12/21 02:18:18 mirmgr Exp $