Manual reduction of filterbank data
The automatic method of reducing
filterbank data works extremely well and you will probably only need
to reduce data manually if you want to use pulsar parameters other
than those in the catalogue or if you have a problem which you want to debug. The aim of this
data reduction is to create what is called an archive.
first step in reducing your data manually is to find some free disk
space to offload your data. You will typically need more than
500Mb. Then you need to set the environment variable tpool to point to
the directory you intend to unload the data into.
As a safety precaution when you are getting data off tape you
should run disk_master, which has sc_td as its
command line argument in this case. This program should be run
from the directory into which the data is going and on the machine to
which the exabyte drive you are using is connected. It controls the
exabyte tape reading program and stops it when there is less than
100Mb of space left on the disk you are writing to and will restart it
when there is greater than 200Mb free.
You are now ready
to extract your data from the exabyte tape. This is done using
sc_td. However you must make sure that you are in the
directory where you want the data to go before you run sc_td,
this can be achieved most simply, if you have set the environment
variable tpool from the step above, by typing cd $tpool. Now
type sc_td, you will be asked a number of questions. The tape
number it requires is the nrst number (usually 0) and typically you
will want to unload all systems with no clipping, no skipping and to
create a summary file. Files will begin to be unloaded, they are
usually quite big so it may be a while before the first one is ready
for you to proceed. As the data is being read off tape the file is
called file.tmp and when it is finished there will be two files
file.dat and file.hdr.
By now you will have a .dat and .hdr file so the next step is to
run fch3. This program takes the raw data and folds it at the
pulse period. Before fch3 can be run there has to be a file
called double.psr in the directory where your data is. A copy of this
file lives in /psr/soft/tas/fch3. If you just type fch3 in the
directory where the data is it will then prompt you to give the filename.
You should leave off the extension. This is the only question you need
to answer as it gets the rest of its information from psrinfo. If it cannot get all the
information it wants from psrinfo then it will fail and this is
discussed in more detail in the errors section. There are a number of output files
associated with fch3 and they will all have a prefix which has a
letter as the first character which is either f or g. The rest of the
prefix is the UT date and time of observation. The majority of these
files are just for bookkeeping, the only important one has the suffix
The next step uses the program ffd2obs which
combines the data in the frequency domain. This program has a large
number of command line inputs which can be seen by just typing
A typical command line would have the form:
ffd2obs -ffdfile f950617_081209.fp3 -fbank -fft -dm 20.74 -nsub
The input file for our application has the .fp3 suffix,
the -fbank tells it that the data is in filtrebank format and the -fft
tells it to do the phase shifting using ffts. The -dm tells it the
dispersion measure to use when adding frequency channels and -nsub
gives the number of groups of channels to output, this should be some
power of two.
The final step is to take the output files from ffd2obs
and convert them into a single archive. This task is performed by
tcombine and the only arguement is the unique name, that is
the combination of f or g and the UT, an example using the same file
as given above is .
The output file has the same root name with the .ar extension.
Note that unlike the automated
processing the raw data files and a number of intermediate files will
remain so it is important that you remove these otherwise you will
quickly run out of diskspace.
All archives should then be moved to the correct pulsar directory
in /psr3/data/timing/ on Kepler. There are two ways to achieve this,
either by moving them by hand, or by using archsort. If you
only have a couple of files then it is probably just as easy to move
them by hand. However when there are a large number of files to move
which is often the case in the $fptmpool directory. Archsort
has two modes, the first is designed to be run on kepler and it
retrieves archives from the $fptmpool directory on Pavo, sorts them
and moves them to the correct directory. This mode is enabled by
on Kepler. The second mode is for when you have accumulated a number
of archives in another directory, such as $fptmpool at Epping. To
enable this simply go to the directory where the archives are and
this will set the current directory to be the
directory from which to move archives.
Back to the Filterbank data reduction page.
Back to the main page.