Find best premium and Free Joomla templates at

Real-Time Data Pipeline

The real-time data processing back-end of the MWA calibrates and images the correlator output, reducing the 5 GB/s input data rate to a manageable size for storage and subsequent processing. It consists of (a) a visibility integrator, (b) a calibrator measurement loop (which includes some foreground prediction and subtraction), (c) an all-sky ionospheric and instrumental calibration systems, and (d) an imaging pipeline which generates images of the sky in Stokes coordinates that have been integrated over several minutes. All of these are software systems that run within the real-time computer.

(a) Visibility Integrator The visibility integrator reduces the time and frequency resolution of the data to an 8 second cadence, while maintaining coherency over the entire sky. All subsequent processing of data must be completed with the 8 seconds.

(b) Calibrator Measurement Loop The real-time calibration of the MWA is naturally broken into three key parts: creation of an all-sky model of ionospheric distortions (updated every 8 seconds), creation of an all-sky model for the antenna beam patterns, and measurement of the data that is used to constrain both these models. Within each cadence, approximately 100 known calibration sources will be used to create the models of the ionosphere and antenna beams. For each calibrator source, the apparent position and flux density are measured directly from the visibilities. The source is then subtracted from the data to reduce its corrupting effect on the other calibrator sources.

(c) All-Sky Instrumental and Ionospheric Calibration The instrumental calibration subsystem determines the gain and polarization calibration for each tile across the whole sky. The apparent flux density measurements from the calibration sources in the previous step are used to update a model for the polarized beam pattern of each antenna. The antenna tiles are modeled based on the physical system, with free parameters for each dipole's complex gain and an overall antenna gain as a function of frequency.

The ionospheric calibration subsystem builds a model of the ionosphere as a phase screen over the entire sky. The offset measurements from calibration sources are used as anchor points in a moving least squares interpolation. Due to the small size of the array relative to the expected ionospheric isoplanatic patch size, we expect that a phase screen model will be an accurate description of the ionosphere in all but the worst conditions.

The ionospheric Faraday rotation subsystem builds a model of the Faraday rotation that the signal will suffer as it passes through the ionosphere. This model will combine the phase screen model from above, an IRI model of the large-scale ionosphere, a model for the local terrestrial magnetic field and total TEC measurements from ground-based GPS receivers.

The results of all calibration measurements and model parameters will be stored so that subsequent data processing can incorporate the measurements and models.

(d) Imaging Pipeline The purpose of the imaging pipeline is to produce images of the sky each 8 seconds ("snapshots") in both instrumental and Stokes coordinates, and to correct for the distorting effects of the ionosphere. The imaging pipeline consists of: interpolating the irregularly sampled visibilities onto a regular grid ("gridding"), generating an image of the sky from the gridded visibilities via FFT, correcting the distortions introduced by ionospheric refraction and converting to Stokes coordinates from the measured instrument polarization.

The visibility gridder subsystem is responsible for gridding and applying instrumental calibration. It takes the visibilities in an ungridded format and places them on a gridded /u,v/ plane. The /u,v/ grid is chosen to cover the primary field of view, and is over-resolved. Gridding involves applying a small gridding convolution function (GCF) consisting of a combination of a regular GCF, to suppress aliasing, a baseline-dependent GCF to correct for any non-coplanar w-terms, and a GCF that corrects for the direction dependent tile gain, which will be unique for each visibility.

After FFT, the data is in image space. The ionospheric distortion is corrected via resampling the image pixels into a celestial coordinate frame. At the same time, the polarized antenna model is used to convert the received instrumental polarization into Stokes coordinates. These corrected images are suitable for averaging over long periods of time, and long (approximately 10 minutes) integrations are accumulated and stored. Because the MWA's antenna tiles see so much of the sky, we store the data in the HEALPIX format, which is suitable for all-sky images.


Real-Time Calibration of the Murchison Widefield Array, Mitchell et al., IEEE Journal of Selected Topics in Signal Processing, 2008, Vol. 2(5), 707-717.

GPUs for data processing in the MWA, Ord et al., in Proc. ADASS 2008, in press.