The datastream process loads baseband data from a source (file, mark5 module, or network socket) into a big ring buffer. The length of this buffer is an integer number of “subintegration” lengths (which is itself an integer number of FFTs). This ring buffer is divided into segments. There are 3 relevant variables controlling the size and slicing of this buffer:
SUBINT_NS: The length of a subintegration in nanoseconds (must be an integer number of FFTs)
DATA_BUFFER_FACTOR: The number of subintegrations in the data buffer
NUM_DATA_SEGMENTS: The number of segments which the data buffer is sliced into
If the correlation has more than one configuration, the one which would create the largest data buffer is used.
The ring buffer is shown in more detail as a picture below:
The Datastream process has one buffer of consequence; the databuffer shown in the image above. There are a number of conflicting requirements which influence a good size for this buffer:
If you start with an appropriate subintegration size, then choosing an appropriate databufferfactor to match the requirements above is usually not too hard. However, figuring out a subint size that is also well-matched to the requirements of the core process can make this procedure somewhat iterative and challenging.