Haystack trip report
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: <Chris.Phillips_at_email.protected>
Date: Wed, 5 Jul 2006 20:12:09 +1000 (EST)
Hi All,
I visited Haystack last week and came back with a number of evlbi related
ideas. Give some of the communication issues lately I thought it best to
send this report to everyone on vlbiobs. Feel free to ignore :-)
----------------------------------------------------------------
Haystack Trip report (20-23 June 2006)
Mark5 compatibility
The Mark5B data format is much simpler than the Mark5a. The data
format of the raw data bits is essentially identical. A simple
translation of LBADR to Mark5b should be easily possible. A realtime
version will also be available. An outline has been proposed to
Haystack to run some tests initially correlating LBADR data at Haystack
with a medium term goal of attempting some eVLBI correlation (not
necessarily at high data rates).
VSI-e
We should attempt to add VSI-e support to the LBADR system on the
medium term (by the end of the year). We probably want to use VSI-e
for *all* eVLBI transfers (realtime eVLBI, remote disk recording and
disk-disk transfers). VSI-e uses the concept of "fill pattern" for
missing data (e.g. when UDP packet loss). The software correlator
would have to be modified to look for this fill pattern and act
appropriately.
Haystack DBE
Haystack are developing a digital backend to replace the Mark4
distribution rack (14 BBCs + formatter etc). This will be relatively
cheap (probably $5000-$15000 range). These are compatible with Mark5b
and our LBADR recorders. These would be a good substitute for new
observatories without an ATNF DAS (New Norcia, NZ), allow stations
with one DAS to upgrade to 1 Gbps recording rates and would allow more
stations to be involved with geodetic experiments (ignoring frequency
coverage issues...). See a separate report on my take of this project.
Mark5b
The Mark5b should be compatible with the ATNF DAS. A report
investigation the cost (effort/money) and benefit to replacing the
LBADR systems. The main benefits would be robustness, international
compatibility and cheaper disks. There will be three main problems:
- Need to buy Conduat Streamstor systems for Swinburne (no need for
full Mark5)
- Moderate software effort needed to interface Streamstor data with
software correlator. It is possible Haystack would do this my making
Linux driver available to make Streamstor data look like normal
filesystem.
- The Streamstor cannot read and write data at the same time. This
will make realtime fringe testing more difficult (not not
impossible. Work arounds could be running dedicated fringe test
experiments in a "burst" mode (record for 5 sec then export 1 sec
of data). Fringe testing during production experiments could be
arrange by exporting small amount of data between scans.
Miscellanea
gigE NICs which support (and which have Linux drivers) hardware
"interrupt mitigation" have improved throughput and less impact on
overall system performance (not necessarily CPU usage).
Ethernet channel bonding can be used to get > 1 Gbps out of a single
host with dual gigE NICs, however this may not work well when
transmitting data through multiple switches.
Linux kernel includes powerful packet level filtering to limit data
rates depending on flexible rules.
Received on 2006-07-05 20:12:44
Date: Wed, 5 Jul 2006 20:12:09 +1000 (EST)
Hi All,
I visited Haystack last week and came back with a number of evlbi related
ideas. Give some of the communication issues lately I thought it best to
send this report to everyone on vlbiobs. Feel free to ignore :-)
----------------------------------------------------------------
Haystack Trip report (20-23 June 2006)
Mark5 compatibility
The Mark5B data format is much simpler than the Mark5a. The data
format of the raw data bits is essentially identical. A simple
translation of LBADR to Mark5b should be easily possible. A realtime
version will also be available. An outline has been proposed to
Haystack to run some tests initially correlating LBADR data at Haystack
with a medium term goal of attempting some eVLBI correlation (not
necessarily at high data rates).
VSI-e
We should attempt to add VSI-e support to the LBADR system on the
medium term (by the end of the year). We probably want to use VSI-e
for *all* eVLBI transfers (realtime eVLBI, remote disk recording and
disk-disk transfers). VSI-e uses the concept of "fill pattern" for
missing data (e.g. when UDP packet loss). The software correlator
would have to be modified to look for this fill pattern and act
appropriately.
Haystack DBE
Haystack are developing a digital backend to replace the Mark4
distribution rack (14 BBCs + formatter etc). This will be relatively
cheap (probably $5000-$15000 range). These are compatible with Mark5b
and our LBADR recorders. These would be a good substitute for new
observatories without an ATNF DAS (New Norcia, NZ), allow stations
with one DAS to upgrade to 1 Gbps recording rates and would allow more
stations to be involved with geodetic experiments (ignoring frequency
coverage issues...). See a separate report on my take of this project.
Mark5b
The Mark5b should be compatible with the ATNF DAS. A report
investigation the cost (effort/money) and benefit to replacing the
LBADR systems. The main benefits would be robustness, international
compatibility and cheaper disks. There will be three main problems:
- Need to buy Conduat Streamstor systems for Swinburne (no need for
full Mark5)
- Moderate software effort needed to interface Streamstor data with
software correlator. It is possible Haystack would do this my making
Linux driver available to make Streamstor data look like normal
filesystem.
- The Streamstor cannot read and write data at the same time. This
will make realtime fringe testing more difficult (not not
impossible. Work arounds could be running dedicated fringe test
experiments in a "burst" mode (record for 5 sec then export 1 sec
of data). Fringe testing during production experiments could be
arrange by exporting small amount of data between scans.
Miscellanea
gigE NICs which support (and which have Linux drivers) hardware
"interrupt mitigation" have improved throughput and less impact on
overall system performance (not necessarily CPU usage).
Ethernet channel bonding can be used to get > 1 Gbps out of a single
host with dual gigE NICs, however this may not work well when
transmitting data through multiple switches.
Linux kernel includes powerful packet level filtering to limit data
rates depending on flexible rules.
Received on 2006-07-05 20:12:44