next up previous
Next: Step 2: Rms-based flagging Up: How Pieflag works Previous: How Pieflag works

Step 1: Amplitude-based flagging

In the first step, Pieflag computes the median visibility amplitude of the reference channel, $x_{\rm b,p}$, for each baseline $b$ and pointing $p$, and the median of the difference to this median, $y_{\rm b,p}$. The use of the median and the median of the difference to the median as a measure for the scatter protects the algorithm from outliers in the reference channel. Both the mean and standard deviation can easily be contaminated by single data points with very high amplitudes, and are therefore not suitable. Pieflag then calculates the difference of each data point in each channel to $x_{\rm b,p}$. If the difference exceeds $ny_{\rm b,p}$, where $n$ is typically around 7, a point is considered suspect and is assigned a ``badness'' value of 1. If the difference exceeds $2ny_{\rm b,p}$, it is deemed certainly bad and is assigned a ``badness'' value of 2. Amplitude-based flagging is most efficient when the data are dominated by receiver noise or in the presence of very weak astronomical signals. In the presence of strong signals from structured sources, $y_{\rm b,p}$ tends to be too high for the algorithm to detect bad data which deviate only little from the source signal (Figure 4). The badness values are converted to flags not before postprocessing (Section 2.3).

If the amplitudes of the visibility amplitudes in the reference channel have a normal distribution, then 68.2% of the data fall within the range of $\pm 1\,\sigma$ of the mean. The median of the difference to the median, however, chooses those 50% of the data that fall within the range of

$x_{\rm b,p}\pm y_{\rm b,p}$, with $y=0.674\sigma$ (this was computed numerically). Hence, in the case of normally distributed amplitudes and $n=7$, $ny_{\rm b,p}$ is equivalent to

$n\times0.674\,\sigma=4.72\,\sigma$.


next up previous
Next: Step 2: Rms-based flagging Up: How Pieflag works Previous: How Pieflag works

Enno Middelberg 2006-03-21
Staff space
Public