If the phase stability is bad during the observation (the atmospheric conditions are poor), the phase of the antenna gains can vary rapidly. A change of tens of degrees or more during the calibrator scan is not uncommon. However the software that determines the antenna gains assumes that the gains are constant during a solution interval. So during periods of poor phase stability, it is often desirable to make the solution interval of the calibration software quite short. While the resultant gains probably track the phase during the calibrator scan, we are more interested in antenna gains for the program source. The best guess at the antenna gains for the program source is an interpolation between the average gain during the calibrator scan. Thus, after determining the gains at a fine time step in the calibrator scan, you would ideally average these gains together to get some representative gain for the whole calibrator scan. This would probably be the best guess you can make (at least as far as correcting the program source is concerned), although in times of truly awful phase stability, its a pretty poor guess (self-calibration will be needed in this case).
There are two ways in which you would want to average your gains, either the vector or scalar average. The general rule is that if you are going to self-calibrate later, then scalar averages are probably the most appropriate. Assuming the variation in gain is purely due to poor phase stability, a scalar average will give you a good estimate of the amplitude of the gain - which is advantageous for self-calibration.
On the other hand, if you are not going to self-calibrate, you are more interested in getting a good estimate of your image at this stage, and less concerned about partially correct gains (the two do not necessarily go hand in hand). If we were to use some average antenna gain, the poor phase stability will cause partial decorrelation in both the program source and calibrator. This will result in apparent reduced flux densities of both of them. Assuming that the decorrelation is approximately the same for both, then we could scale up the program source by the decorrelation that we note in the calibrator. This could be achieved by vector averaging the antenna gains.
You may have guessed from the frequent use of the word ``would'' in the discussion above that AIPS cannot do all that we would want here. It does not offer a useful gain averaging task. The task SNSMO can smooth gains, but it can only do a scalar average. The alternative in AIPS is to average the data rather than the gains. This is not ideal because the averaged visibilities may have non-closing errors in them (i.e., the mathematical model upon which the calibration procedure is based may be invalidated). However, in practice, it seems to work reasonably well. Thus, ATCALIB offers you vector or scalar averaging of data rather than vector or scalar averaging of the gains.
In summary, you should keep the solution interval at a scan average {solint=0, and use vector averaging for data with good phase stability or if you not going to be able to self-calibrate your data (see § 17). Use scalar averaging if the phase stability fis poor and you are going to be able to self-calibrate.