| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| difx:nativemk5 [2010/11/19 05:50] – walterbrisken | difx:nativemk5 [2015/10/21 10:08] (current) – external edit 127.0.0.1 |
|---|
| | |
| ===== How to correlate directly off Mark5 modules ===== | ===== How to correlate directly off Mark5 modules ===== |
| |
| ==== 1. Linux requirements ==== | ==== 1. Linux requirements ==== |
| |
| Compatibility of the Conduant streamstor drivers severely limits the choice of Linux kernels. The supported kernel versions depend on the exact version of the driver (the SDK version number) that is used. SDK version 8.2 or later are suggested as earlier versions have known stability issues related to the way the streamstor cards are accessed by DiFX. As of this writing (Nov 18, 2010) only 32-bit kernels are supported. See [[mixingarchitectures|mixing architectures]] to see how to configure a cluster to support a mixture of 32 bit and 64 bit members of a cluster running mpifxcorr. Note that mixing different streamstor SDK versions across the cluster is possible, but for each a separately compiled version of the mpifxcorr binary will be required due to differences in library requirements. Differences between major SDK releases (e.g., SDK8 and SDK9) usually also have API-level changes. Currently mpifxcorr can be compiled against either SDK8 or SDK9 series. | Compatibility of the Conduant StreamStor drivers severely limits the choice of Linux kernels. The supported kernel versions depend on the exact version of the driver (the SDK version number) that is used. SDK version 8.2 or later are suggested as earlier versions have known stability issues related to the way the StreamStor cards are accessed by DiFX. As of this writing (Dec 8, 2011) only 32-bit kernels are supported. See [[mixingarchitectures|mixing architectures]] to see how to configure a cluster to support a mixture of 32 bit and 64 bit members of a cluster running ''mpifxcorr''. Note that mixing different StreamStor SDK versions across the cluster is possible, but for each a separately compiled version of the ''mpifxcorr'' binary will be required due to differences in library requirements. Differences between major SDK releases (e.g., SDK8 and SDK9) usually also have API-level changes. Currently ''mpifxcorr'' can be compiled against either SDK8 or SDK9 series. There are reports (from RJC: specific failure is on kernel 2.6.26 and SDK 9.0) that Linux Kernel versions greater than 2.6.18 interact badly with the StreamStor/Jungo driver, resulting in garbage data returned for read sizes greater than 8 MB. |
| |
| ==== 2. Installation of Conduant's streamstor Software Development Kit (SDK) ==== | ==== 2. Installation of Conduant's streamstor Software Development Kit (SDK) ==== |
| ==== 3. Installing a package-config file for streamstor ==== | ==== 3. Installing a package-config file for streamstor ==== |
| |
| For ''mpifxcorr'' and ''mk5daemon'' to find the libraries and include files for the streamstor SDK, it is most convenient to place a package-config (.pc) file in a location identified by environment variable ''PKG_CONFIG_PATH''. Note that the streamstor libraries must be in the ''LD_LIBRARY_PATH'' or in a system library directory for run-time linking. Two examples follow: | For ''mpifxcorr'' and ''mk5daemon'' to find the libraries and include files for the streamstor SDK, it is most convenient to place a package-config (.pc) file in a location identified by environment variable ''PKG_CONFIG_PATH''. Note that the StreamStor libraries must be in the ''LD_LIBRARY_PATH'' or in a system library directory for run-time linking. Two examples follow: |
| |
| Example streamstor.pc for SKD8.2 | Example streamstor.pc for SKD8.2 |
| ==== 4. Compilation of ''mpifxcorr'' ==== | ==== 4. Compilation of ''mpifxcorr'' ==== |
| |
| Compilation of ''mpifxcorr'' should proceed as usual. The configuration process should find the streamstor.pc file and properly configure the compilation for native mark5. You should see ''nativemk5.cpp'' being correlated as a result. | Compilation of ''mpifxcorr'' should proceed as usual. The configuration process should find the streamstor.pc file and properly configure the compilation for native mark5. You should see ''nativemk5.cpp'' being compiled as a result. |
| |
| ==== 5. Dealing with Mark5 Module directories ==== | ==== 5. Dealing with Mark5 Module directories ==== |
| |
| Mark5 modules do not contain a conventional filesystem and due to the potential of failure in decoding modules and the fact that decoding the scans can be time consuming, the reading of module directories to ''.dir'' files is handled outside the correlation process. These ''.dir'' files are stored in the directory pointed to by the ''$MARK5_DIR_PATH$'' environment variable. The program ''mk5dir'', found in the ''mk5daemon'' package, performs the decoding of the modules. This can be run manually on each Mark5 module at the command line: | Mark5 modules do not contain a conventional file system and due to the potential of failure in decoding modules and the fact that decoding the scans can be time consuming, the reading of module directories to ''.dir'' files is handled outside the correlation process. These ''.dir'' files are stored in the directory pointed to by the ''$MARK5_DIR_PATH$'' environment variable. The program ''mk5dir'', found in the ''mk5daemon'' package, performs the decoding of the modules. This can be run manually on each Mark5 module at the command line: |
| |
| <code sh> | <code sh> |
| |
| The ''.dir'' file contains a hash code at the top. If the hash code matches that of the installed module (implying that no new scans have been added and that all existing scans are unchanged) then the directory file is not updated. | The ''.dir'' file contains a hash code at the top. If the hash code matches that of the installed module (implying that no new scans have been added and that all existing scans are unchanged) then the directory file is not updated. |
| | |
| | Modules containing the new directory format [[http://www.haystack.mit.edu/tech/vlbi/mark5/mark5_memos/046.pdf|Mark5 Memo 46]] can take advantage of a mode where decoding of each scan is not required. The ''--fast'' (or ''-F'') option tells ''mk5dir'' to take format and timing data straight from the on-module directory with a large decrease in time to decode. This "shortcut" does eliminate a useful sanity check of the module's health that can save time at correlation time. |
| | |
| | A "signature" is calculated based on the binary composition of the on-disk header. It is similar to a checksum and is used to determine if the ''.dir'' file is current with the module. This signature is stored in the 4th column of the first line. It will typically be an 8 to 10 digit number. The signature of an empty directory is 1. This fact can be useful if you want to modify a previously extracted directory to work on a module that somehow had its on-disk directory erased. |
| | |
| | When a module's directory is read, ''mk5dir'' will, if deemed safe, change the Disk Module State (DMS) to ''played''. This can be dangerous if there is an SDK mismatch between the unit used to record the module and the one used to play back the module. Unfortunately there is no way to determine the former by looking at the module, though some inferences, of varying degree of certainty, can be made. The setting of DMS to ''played'' can be prevented either by setting the environment variable DEFAULT_DMS_MASK to 5 (this is a binary bit mask: 4 = "erased", 2 = "played", 1 = "recorded"), or by specifying one of the dms option flags to ''mk5dir'' on the command line; run ''mk5dir --help'' for more details on these. |
| | |
| | A module directory can be tested for common errors with ''checkdir'' |
| | |
| | <code sh> |
| | checkdir NRAO+321 |
| | </code> |
| | |
| | Run ''checkdir'' with ''--help'' to get additional usage options. |
| |
| ==== 6. Running a correlator job ==== | ==== 6. Running a correlator job ==== |
| |
| The module to be installed (NRAO+293 in the above example) can be installed in either bank of the Mark5 unit. | The module to be installed (NRAO+293 in the above example) can be installed in either bank of the Mark5 unit. |
| | |
| | |
| | Cranky Mark5 modules can be frustrating at a correlator. A module that shows consistent problems can be put into "realtime" mode by changing the ''NORMAL'' to ''RT'' at the top of the directory file. This often, but not always, results in better handing of imperfect modules. |
| |
| ==== 7. mk5daemon and extra functionality ==== | ==== 7. mk5daemon and extra functionality ==== |
| |
| |
