User Tools

Site Tools


difx:benchmarking

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
difx:benchmarking [2015/10/21 10:08]
127.0.0.1 external edit
difx:benchmarking [2018/09/06 18:03] (current)
walterbrisken
Line 29: Line 29:
  
 A tool called ''difxspeed'' was developed that tests DiFX performance using the 'FAKE' data source mode.  In this mode the entire correlator is configured as it would be for normal processing, but instead of reading data from a file, Mark5 unit, or from the network, junk data will be produced at a datastream node with just enough formatting to fool the rest of the system.  Everything else will proceed as it normally would, yielding fairly accurate performance ratings.  A file similar to a simplified ''.v2d'' file should be prepared.  In this file many typical ''.v2d'' parameters are accepted.  Unlike the ''.v2d'' file syntax, there are no ANTENNA, SETUP, ... blocks (sections within curly braces).  Instead all parameters are to be specified at the root level.  5 parameters are required: ''cores'' is a prioritized list of hostnames to run the core processes, ''nCore'' is the number of core processes to start, ''datastreams'' is a prioritized list of hostnames on which to run datastream processes, ''nAnt'' is the number of antennas, and ''antennas'' is a prioritized list of antennas to include in the correlation.  Note that most simple parameters (including ''nCore'' and ''nAnt'') can take a list of values.  ''difxspeed'' will run every possible combination of the supplied value lists.  This can be used to look at performance as a function of processing cores being used, values of certain buffer sizes, ... A tool called ''difxspeed'' was developed that tests DiFX performance using the 'FAKE' data source mode.  In this mode the entire correlator is configured as it would be for normal processing, but instead of reading data from a file, Mark5 unit, or from the network, junk data will be produced at a datastream node with just enough formatting to fool the rest of the system.  Everything else will proceed as it normally would, yielding fairly accurate performance ratings.  A file similar to a simplified ''.v2d'' file should be prepared.  In this file many typical ''.v2d'' parameters are accepted.  Unlike the ''.v2d'' file syntax, there are no ANTENNA, SETUP, ... blocks (sections within curly braces).  Instead all parameters are to be specified at the root level.  5 parameters are required: ''cores'' is a prioritized list of hostnames to run the core processes, ''nCore'' is the number of core processes to start, ''datastreams'' is a prioritized list of hostnames on which to run datastream processes, ''nAnt'' is the number of antennas, and ''antennas'' is a prioritized list of antennas to include in the correlation.  Note that most simple parameters (including ''nCore'' and ''nAnt'') can take a list of values.  ''difxspeed'' will run every possible combination of the supplied value lists.  This can be used to look at performance as a function of processing cores being used, values of certain buffer sizes, ...
 +
 +Detailed information on [[difxspeed]] can be found.
  
 Some additional information, including access to the difxspeed program itself, can be found at the NRAO wiki [[https://safe.nrao.edu/wiki/bin/view/HPC/UsnoDifxBenchmarking]].  This information will eventually be moved to the CIRA wiki. Some additional information, including access to the difxspeed program itself, can be found at the NRAO wiki [[https://safe.nrao.edu/wiki/bin/view/HPC/UsnoDifxBenchmarking]].  This information will eventually be moved to the CIRA wiki.
difx/benchmarking.txt · Last modified: 2018/09/06 18:03 by walterbrisken