User Tools

Site Tools


correlator:lbafilecheck

lbafilecheck.py

this is a simple script to create and check the telescope filelists and to create a machines, threads and run file. It assumes you want to use all the nodes listed in $CORR_HOSTS (normally this is /home/corr/corr_hosts.txt).

lbafilecheck.py v311a.datafiles

An example .datafiles file:

v311a *.lba
at=cuppa01:/exports/xraid01/r_1/v311a/atca/
cd=cuppa02:/exports/xraid02/l_1/v311a/
ho=cuppa02:/exports/xraid02/r_1/v311a-Ho/
mp=cuppa04:/exports/xraid04/l_1/v311a-Mp/
pa=cuppa16:/mnt/disk1/v311a-pk/ /mnt/disk2/v311a-pk/

the first value on the first line must be the experiment name. The optional following entries on the first line specify global pattern matches that will be appended to the patterns specified in the following lines. You may have multiple, space separated, global patterns. The remaining lines are a telescope id, followed by '=', followed by the node name followed by a ':' followed by a space separated list of search patterns that will be used to find data files for that telescope id (with the the global pattern appended when specified). The trailing slashes are important as the patterns formed by concatenating the telescope patterns with the global pattern will be passed directly to 'ls' to produce the file list. Text following a '#' is interpreted as a comment.

More complicated patterns can be useful to select a subset of files for clock jobs, or jobs that do not start at the beginning. But typically '*' will suffice.

correlator/lbafilecheck.txt · Last modified: 2016/04/21 16:23 by cormac