User Tools

Site Tools


difx:benchmarking

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
difx:benchmarking [2015/10/21 10:08]
127.0.0.1 external edit
difx:benchmarking [2018/09/06 18:03] (current)
walterbrisken
Line 29: Line 29:
  
 A tool called ''​difxspeed''​ was developed that tests DiFX performance using the '​FAKE'​ data source mode.  In this mode the entire correlator is configured as it would be for normal processing, but instead of reading data from a file, Mark5 unit, or from the network, junk data will be produced at a datastream node with just enough formatting to fool the rest of the system. ​ Everything else will proceed as it normally would, yielding fairly accurate performance ratings. ​ A file similar to a simplified ''​.v2d''​ file should be prepared. ​ In this file many typical ''​.v2d''​ parameters are accepted. ​ Unlike the ''​.v2d''​ file syntax, there are no ANTENNA, SETUP, ... blocks (sections within curly braces). ​ Instead all parameters are to be specified at the root level. ​ 5 parameters are required: ''​cores''​ is a prioritized list of hostnames to run the core processes, ''​nCore''​ is the number of core processes to start, ''​datastreams''​ is a prioritized list of hostnames on which to run datastream processes, ''​nAnt''​ is the number of antennas, and ''​antennas''​ is a prioritized list of antennas to include in the correlation. ​ Note that most simple parameters (including ''​nCore''​ and ''​nAnt''​) can take a list of values. ​ ''​difxspeed''​ will run every possible combination of the supplied value lists. ​ This can be used to look at performance as a function of processing cores being used, values of certain buffer sizes, ... A tool called ''​difxspeed''​ was developed that tests DiFX performance using the '​FAKE'​ data source mode.  In this mode the entire correlator is configured as it would be for normal processing, but instead of reading data from a file, Mark5 unit, or from the network, junk data will be produced at a datastream node with just enough formatting to fool the rest of the system. ​ Everything else will proceed as it normally would, yielding fairly accurate performance ratings. ​ A file similar to a simplified ''​.v2d''​ file should be prepared. ​ In this file many typical ''​.v2d''​ parameters are accepted. ​ Unlike the ''​.v2d''​ file syntax, there are no ANTENNA, SETUP, ... blocks (sections within curly braces). ​ Instead all parameters are to be specified at the root level. ​ 5 parameters are required: ''​cores''​ is a prioritized list of hostnames to run the core processes, ''​nCore''​ is the number of core processes to start, ''​datastreams''​ is a prioritized list of hostnames on which to run datastream processes, ''​nAnt''​ is the number of antennas, and ''​antennas''​ is a prioritized list of antennas to include in the correlation. ​ Note that most simple parameters (including ''​nCore''​ and ''​nAnt''​) can take a list of values. ​ ''​difxspeed''​ will run every possible combination of the supplied value lists. ​ This can be used to look at performance as a function of processing cores being used, values of certain buffer sizes, ...
 +
 +Detailed information on [[difxspeed]] can be found.
  
 Some additional information,​ including access to the difxspeed program itself, can be found at the NRAO wiki [[https://​safe.nrao.edu/​wiki/​bin/​view/​HPC/​UsnoDifxBenchmarking]]. ​ This information will eventually be moved to the CIRA wiki. Some additional information,​ including access to the difxspeed program itself, can be found at the NRAO wiki [[https://​safe.nrao.edu/​wiki/​bin/​view/​HPC/​UsnoDifxBenchmarking]]. ​ This information will eventually be moved to the CIRA wiki.
difx/benchmarking.txt · Last modified: 2018/09/06 18:03 by walterbrisken