Manual reduction of filterbank data

The automatic method of reducing filterbank data works extremely well and you will probably only need to reduce data manually if you want to use pulsar parameters other than those in the catalogue or if you have a problem which you want to debug. The aim of this data reduction is to create what is called an archive.

  • The first step in reducing your data manually is to find some free disk space to offload your data. You will typically need more than 500Mb. Then you need to set the environment variable tpool to point to the directory you intend to unload the data into.


  • As a safety precaution when you are getting data off tape you should run disk_master, which has sc_td as its command line argument in this case. This program should be run from the directory into which the data is going and on the machine to which the exabyte drive you are using is connected. It controls the exabyte tape reading program and stops it when there is less than 100Mb of space left on the disk you are writing to and will restart it when there is greater than 200Mb free.


  • You are now ready to extract your data from the exabyte tape. This is done using sc_td. However you must make sure that you are in the directory where you want the data to go before you run sc_td, this can be achieved most simply, if you have set the environment variable tpool from the step above, by typing cd $tpool. Now type sc_td, you will be asked a number of questions. The tape number it requires is the nrst number (usually 0) and typically you will want to unload all systems with no clipping, no skipping and to create a summary file. Files will begin to be unloaded, they are usually quite big so it may be a while before the first one is ready for you to proceed. As the data is being read off tape the file is called file.tmp and when it is finished there will be two files file.dat and file.hdr.


  • By now you will have a .dat and .hdr file so the next step is to run fch3. This program takes the raw data and folds it at the pulse period. Before fch3 can be run there has to be a file called double.psr in the directory where your data is. A copy of this file lives in /psr/soft/tas/fch3. If you just type fch3 in the directory where the data is it will then prompt you to give the filename. You should leave off the extension. This is the only question you need to answer as it gets the rest of its information from psrinfo. If it cannot get all the information it wants from psrinfo then it will fail and this is discussed in more detail in the errors section. There are a number of output files associated with fch3 and they will all have a prefix which has a letter as the first character which is either f or g. The rest of the prefix is the UT date and time of observation. The majority of these files are just for bookkeeping, the only important one has the suffix .fp3.


  • The next step uses the program ffd2obs which combines the data in the frequency domain. This program has a large number of command line inputs which can be seen by just typing ffd2obs -h. A typical command line would have the form:

    ffd2obs -ffdfile f950617_081209.fp3 -fbank -fft -dm 20.74 -nsub 16

    The input file for our application has the .fp3 suffix, the -fbank tells it that the data is in filtrebank format and the -fft tells it to do the phase shifting using ffts. The -dm tells it the dispersion measure to use when adding frequency channels and -nsub gives the number of groups of channels to output, this should be some power of two.


  • The final step is to take the output files from ffd2obs and convert them into a single archive. This task is performed by tcombine and the only arguement is the unique name, that is the combination of f or g and the UT, an example using the same file as given above is .

    tcombine f950617_081209

    The output file has the same root name with the .ar extension. Note that unlike the automated processing the raw data files and a number of intermediate files will remain so it is important that you remove these otherwise you will quickly run out of diskspace.


  • All archives should then be moved to the correct pulsar directory in /psr3/data/timing/ on Kepler. There are two ways to achieve this, either by moving them by hand, or by using archsort. If you only have a couple of files then it is probably just as easy to move them by hand. However when there are a large number of files to move which is often the case in the $fptmpool directory. Archsort has two modes, the first is designed to be run on kepler and it retrieves archives from the $fptmpool directory on Pavo, sorts them and moves them to the correct directory. This mode is enabled by simply typing,

    archsort

    on Kepler. The second mode is for when you have accumulated a number of archives in another directory, such as $fptmpool at Epping. To enable this simply go to the directory where the archives are and type,

    archsort .

    this will set the current directory to be the directory from which to move archives.
    Back to the Filterbank data reduction page.
    Back to the main page.