User Tools

Site Tools


difx:difxspeed

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
difx:difxspeed [2018/09/06 18:38]
walterbrisken
difx:difxspeed [2018/09/06 20:46] (current)
walterbrisken
Line 20: Line 20:
 vex = dq224.vex</​code>​ vex = dq224.vex</​code>​
  
-This file must end with extension ​=.difxspeed.  This program takes a given =.vex= ​file, creates a series of =.v2d= files, one for each potential configuration as described by the parameter options, creates DiFX file sets, and executes them.  For the above example, DiFX will be invoked 49 times. ​ The first time is a "​dummy" ​run, used to make sure all of the equipment is "​warmed up" (and this can make a difference),​ and then there is a run followed by each of the 48 combinations of =xmacLength=,​ =strideLength=,​ and =numBufferedFFTs= suggested by the file.  The files are always run in "​fake"​ mode where no data is actually read, rather randomly generated data is pushed through the system. ​ Most of the parameters in this file match those of a =.v2d= file.+This file must end with extension ​''​.difxspeed'' ​.  This file can be run with command:
  
-Running this set of benchmarks will produce an output file (ending in =.out= ) which may resemble:+<​code>​difxspeed dq224.difxspeed</​code>​
  
-<verbatim>+This file must end with extenThis program takes a given ''​.vex''​ file, creates a series of ''​.v2d''​ files, one for each potential configuration as described by the parameter options, creates DiFX file sets, and executes them.  For the above example, DiFX will be invoked 49 times. ​ The first time is a //dummy// run, used to make sure all of the equipment is "​warmed up" (and this can make a difference),​ and then there is a run followed by each of the 48 combinations of ''​xmacLength'',​ ''​strideLength'',​ and ''​numBufferedFFTs''​ suggested by the file.  By default, correlation is always run in //fake// mode where no data is actually read, rather randomly generated data is pushed through the system. ​ See below for running with real data.  Most of the parameters in this file match those of a ''​.v2d''​ file. 
 + 
 +Running this set of benchmarks will produce an output file (ending in ''​.out''​ ) which may resemble: 
 + 
 +<code>
 # Cluster definition: # Cluster definition:
 # datastreams = node-2 node-2 node-2 node-2 node-2 # datastreams = node-2 node-2 node-2 node-2 node-2
Line 94: Line 98:
 32 32 8  138.268167 0.945924 ​ 138.183000 138.132000 137.222000 139.771000 137.160000 139.141000 ​ # dq2-046 32 32 8  138.268167 0.945924 ​ 138.183000 138.132000 137.222000 139.771000 137.160000 139.141000 ​ # dq2-046
 64 32 8  130.166500 1.286886 ​ 128.630000 131.164000 130.814000 131.897000 130.106000 128.388000 ​ # dq2-047 64 32 8  130.166500 1.286886 ​ 128.630000 131.164000 130.814000 131.897000 130.106000 128.388000 ​ # dq2-047
-128 32 8  125.817333 1.249397 ​ 124.964000 125.225000 125.255000 124.508000 128.090000 126.862000 ​ # dq2-048</​verbatim>​+128 32 8  125.817333 1.249397 ​ 124.964000 125.225000 125.255000 124.508000 128.090000 126.862000 ​ # dq2-048</​code> 
 + 
 +The above file is actually the result of running the same benchmarking script 6 times. ​ The run times are extracted from the ''​.difxlog''​ files which are not erased after each run, this system can be used to construct a history of benchmarking results. ​ The first set of columns in this file are the three parameters that are allowed to vary in this case.  The next two are the average and RMS run times, and finally the individual trial run times are printed. ​ A comment at the end of each line indicates the DiFX file set associated with each run. 
 + 
 +===== Running with non-fake data ===== 
 + 
 +If performance of the entire correlation process, not just the number-crunching,​ is desired, two changes are needed to the file.  First, fake mode needs to be disabled through inclusion of ''​fake=false''​ in the ''​.difxspeed''​ file.  The second is including the antenna and datastream information,​ which is done much as its is in a regular correlation. ​ The ANTENNA sections (and DATASTREAM sections, if used) can be explicitly transferred from the ''​.difxspeed''​ file.  To do this, include a line in the ''​.difxspeed''​ file consisting entirely of ''​-''''​-''​ (two hyphens). ​ All lines of text below this are copied ​verbatim ​to the bottom of the ''​.v2d''​ file. 
 + 
 +An shortened example demonstrating non-fake data correlation follows: 
 + 
 +<​code>​ 
 +datastreams = node-2,​node-3 
 +cores = node-4,​node-5 
 +antennas = BR,FD 
 + 
 +nAnt = 4 
 +nCore = 2 
 +nThread = 3 
 +tInt = 1 
 +specRes = 0.125 
 +fftSpecRes = 0.125 
 +xmacLength = 16,​32,​64,​128 
 +strideLength = 8,16,32 
 +numBufferedFFTs = 1,2,4,8 
 + 
 +vex = dq224.vex 
 + 
 +fake = false 
 + 
 +-- 
 + 
 +ANTENNA BR { vsn=NRAO+234 } 
 +ANTENNA FD { file=dq224_fd_01.mark5b }</code>
  
-The above file is actually the result of running the same benchmarking script 6 times. ​ The run times are extracted from the =.difxlogfiles which are not erased after each run, this system can be used to construct a history ​of benchmarking results. ​ The first set of columns in this file are the three parameters that are allowed to vary in this case.  The next two are the average and RMS run times, and finally the individual trial run times are printed. ​ A comment at the end of each line indicates the DiFX file set associated with each run.+===== Some notes of interest =====
  
-Some notes of interest: +  ​* Once in a while (e.g., dq2-043 in the above output) the full runtime seems not to be captured. ​ This is being investigated. ​ The problem appears to be in the capture of the multicast messages by the ''​difxlog'' ​program. 
-  ​* Once in a while (e.g., dq2-043 in the above output) the full runtime seems not to be captured. ​ This is being investigated. ​ The problem appears to be in the capture of the multicast messages by the =difxlogprogram. +  * The files produced invoke ​''​singleScan=false''​, so a multiple scan ''​.vex'' ​file can be used. 
-  * The files produced invoke ​"singleScan=false", so a multiple scan =.vexfile can be used. +  * Things will work poorly (if at all) if more than 1 setup is used within the ''​.vex'' ​file.
-  * Things will work poorly (if at all) if more than 1 setup is used within the =.vexfile.+
  
difx/difxspeed.1536223101.txt.gz · Last modified: 2018/09/06 18:38 by walterbrisken