Tarballs of difx are not presently available - in order to get access to the latest version of DiFX, you need to download it from SVN.

First, if you have not already, please join the DiFX-Users group - to do this, go to https://listmgr.nrao.edu/mailman/listinfo/difx-users and fill in the form.

Two separate installation methods are shown below. First is the classic, well tested install-difx based installer. The second is a newer version that is probably less complete, but offers multi-architecture building and management.

Create an SVN area (somewhere with network access, obviously) and type:

svn co https://svn.atnf.csiro.au/difx/virtualtrunk

This should check out the trunk versions of all the packages that comprise the DiFX ecosystem. This includes mpifxcorr (the actual correlator), other necessary correlator plug-ins and a host of useful auxiliary programs. To compile, you can run install-difx as described below.

At the time of writing, the current stable version of DiFX is DiFX-2.6.2. You should look on https://svn.atnf.csiro.au/difx/master_tags/ to see what the latest version is, and check the difx-users mailing archive to make sure that the highest version number you see there has been declared stable. Then, to check out (for example) DiFX 2.6.2 type:

svn co https://svn.atnf.csiro.au/difx/master_tags/DiFX-2.6.2

If your username is different between your login and your svn account use the option –username with your SVN username. This should check out the entire DiFX-2.6.2 package, which contains everything you need to run the software correlator.

A subversion password is no longer required for read access. If you want write access you will need to fill-in the form at http://svn.atnf.csiro.au/cgi-bin/svnpasswd, requesting DIFX access. This will also give you access to the trac bug submission and source browsing tool: https://svn.atnf.csiro.au/trac/difx/browser/.

Please also contact Chris Phillips (chris.phillips AT csiro.au) with a short justification and introduction if you don't know Chris, or get someone to “vouch” for you (e.g. an existing DIFX developer).

The correlator needs an optimized vector library, which by default (and is currently the only supported option) is Intel Performance Primitives (IPP). You can download a trial, or buy it (its pretty cheap) from https://software.intel.com/en-us/intel-ipp, or, if you qualify, get it for free from https://software.intel.com/en-us/articles/free-ipp.

DiFX adds support for new IPP versions on a regular basis, but very new versions of IPP may not yet have support. Many people have versions stretching back as far as 4.x. Unfortunately, the names of the libraries which it is necessary to link against change from version to version. We have taken care of the majority of the changes (which occur with 32 bit vs 64 bit libraries) in the automake setup for DiFX. However, you may notice when compiling that the default linker line is not correct for your local installation. A problem which has previously cropped up is the demand for “-liomp5” with ipp6.0, for instance.

Recently, a better solution to the IPP configuration problem has been developed by Walter B, who put together a python script called genipppc. This generates a package-config (.pc) file suitable for the IPP library. To use this script, first install IPP. Then go to svn_checkout_area/setup/, where svn_checkout_area is where you ran the SVN checkout from. In this directory you will find genipppc - run it with one argument, which is the path to the base of your IPP install. This will generate a single file called ipp.pc. Copy it to $DIFXROOT/lib/pkgconfig/ (you may need to create this directory). From this point on, as long as $DIFXROOT/lib/pkgconfig/ is in your $PKG_CONFIG_PATH (which it will be if you use setup.bash/setup.csh) all the DiFX programs will know where to find the IPP include and library files. If you have any problems with linking against IPP when compiling DiFX, you may need to edit this ipp.pc file (for instance, to add -liomp5, or to change the name of a path. Please let the difx-users mailing list know if you have problems, and we will try to improve genipppc so that the configuration works properly for the next user).

Intel has a yum repo for it; add this to your /etc/yum.repos.d/intelproducts.repo and you may be able to get it. (You may need to wander through their registration pages to get to the same thing downloaded.)

[intel-ipp-repo]
name=Intel(R) Integrated Performance Primitives
baseurl=https://yum.repos.intel.com/ipp
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB

after which you can install the beast with a simple

yum install intel-ipp-64bit-2019.1-053.x86_64

(2019.1 is the latest version, and it compiles, but it has not been thoroughly tested. 2018.X or 2017.X are safer.)

The correlator uses an implementation of Message Passing Interface (MPI) to shoot data around. Most linux systems will probably already have an implementation of MPI installed - try typing “which mpicc” to see if you have one installed. If not, two well-tested systems are mpich1 and openmpi.

Now, you should see all the components of DiFX laid out beneath you. The major components are in mpifxcorr/ (the actual correlator), libraries/ (which includes libraries for reading/writing data, logging, configuring the correlator etc), and applications/ (where other applications like vex2difx, difx2fits etc live). The DiFX project uses GNU autotools, which should be familiar to anyone who uses linux regularly. At the top level is a README.txt file which describes how to install the entire correlator.

You need to set some local paths in the setup scripts and run install-difx. If you have checked out a tagged release, these files will be in the top directory of the difx checkout code. If you have checked out the full difx repository. These files are in the setup subdirectory (and install-difx must be run from there).

• Edit setup.bash or setup.csh to set various paths as required. If you have done full subversion checkout, set DIFX_VERSION to be either trunk or the name of the branch you want to compile
• source setup.bash (or setup.csh)
• Probably source the setup files from you login scripts
• ./install-difx

If you want some non-standard perl utilities to also be installed, run ./install-difx –perl

This will take some time to run, but barring unforeseen problems will leave you with a happily installed correlator! For a full list of install-difx options, run

 ./install-difx --help

In the unhappy event of install problems, see the Troubleshooting section below. An alternate installation process is described here.

difxbuild is a rather new building and source management system for DiFX. Documentation for this is currently hosted on the NRAO wiki at https://safe.nrao.edu/wiki/bin/view/HPC/UsnoDifxInstallation. The documentation will be moved here at some point. The program is a single self-contained python program that can be found at this NRAO wiki or in the DiFX subversion repository in https://svn.atnf.csiro.au/difx/sites/NRAO/trunk/bin. Typically the later will be more up to date. In addition to the documentation found on the NRAO wiki there is in-line help which is accessible by running:

difxbuild -d

Notes:

• difx_db and its dependency cx_Oracle are VLBA operations specific.
• package calcif2 used to be called job2difx

DiFX package dependencies for various distributions

Fedora

Debian

Ubuntu

Mac OSX

A newer alternative is difxcalc, built on Calc11 which does not require calcserver. This is much easier to get going.

In order to generate geometric models to use in the correlator, you will need to start the calcServer program (unless you are at an institute that already has a calcserver installation). In order to do this, run startCalcServer as root (or with su). Once you have done this, you should check it is running using checkCalcserver. Make sure to set the $CALC_SERVER environment variable to an appropriate value (the machine name where you will run the calc server). You may also wish to (as root, or with su) run “cp directory_to_install_DiFX/bin/calcserver /etc/init.d” and have this file be run at bootup (so you don't have to restart the calcserver manually after every reboot).

• DiFX continues to fail to compile after fixing some problem (such as missing dependency)

Run ./difx-install –reconfig

• DiFX wants to use gfortran, I have g77

Run ./difx-install –g77