Harri Hohti, FMI:

Suboptimal compositing algorithms

The compositing of radar network data has traditionally been done by combining PPI or CAPPI products of individual radars. Several different principles of choosing the data, like nearest radar, closest to earth, maximum, average and different combinations of these have been used in compositing algorithms with varying success.

Totally new problems will appear if something else than horizontal products, e.g. composited cross sections are needed, but those are left out of this study.

Even in case of optimally corrected cartesian products, one problem remains: How to choose between two or more geometrically equal, but different reflectivity values, if quality of these values are not known? An inevitable consequence of this is that there must be lots of cases where quality parameters dominates the data selection between two or more radars, not the geometry.

Problems related to cartesian products

The "factory made" cartesian products themselves have some buildt-in problems which causes information loss or bias:

- Near the radar the coordinate conversion itself causes the loss of data due to cartesian resolution being coarser than the the polar one. This is true with PPI and CAPPI products both.

- The method used to interpolate data between elevation angles affects the data when CAPPIs are used as the cartesian products. This causes a kind of inversion problem; it isn't known if the data value in some pixel is the actual measured one or some interpolated/extrapolated one. So the reliability of the data has no measure.

- Quality indicators and correction terms derived from effects of interpolation, attenuation, beam blocking etc are perhaps taken into account when the cartesian product is constructed inside the radar software, but these can't be extracted from the product itself, or the data format does not allow inclusion of these.

Conclusion: quality indicator fields are necessary when selecting/  calculating the optimal values of data from two or more radars.

Problems related to selection rules

Even having the best possible cartesian products available there is still problems with each of compositing principles. In the following list it is assumed that optimal estimation of ground precipitation is the goal of compositing.

To use these rules there must at least information of elevation angles and radar heights available, so that the height of measurement at each point can be calculated. It is also assumed that calibration of radars are equal, which is usually not true.

- closest to earth:  The pixel closest to earth doesn't necessarily contain the most qualified data. Especially clutter and beam blocking can easily contaminate  the data from lowest elevation angle, and attenuation due precipitation can cause underestimation of reflectivity if not corrected.

- nearest radar: Works correctly only when scanning strategies of radars are equal and radars are situated at the same height.

- maximum:  This rule has several problems. It works considerably fine only if some of the radars in network suffers of serious beam blocking. Otherwise a bad choice because the vertical profile of reflectivity is not constant. Overhanging precipitation is the worst problem when using this rule, and bright band can also cause overestimation of ground precipitation.

- average: Usually the worst choice, because output values are always biased. Especially in case of small but intesive convection cells both distance dependent   beam height and difference in resolution causes smoothing and leads to serious underestimation of precipitation.

Requirements due the users of data

It is also important to realize that composite made by using one, even good algorithm may not necessarily fulfil the needs of the end-user of the data. Even the "ground truth" product can have totally different requirements. For example, for hydrological purposes there is no need for nice pictures, only as correct as possible data is important. On the other hand, a radar composite shown on commercial TV-channel has almost opposite requirements;  the picture must be just fine looking, nice and smooth, without any need for "correct" values of single pixels.

So there should be a possibility for fast production of composites which
obeys several different requirements. This means that design, tuning and
implementation of new algorithms or selection rules should be flexible.

Possible solutions

One possible solution to the problem developed at FMI uses polar volumes directly. It is based on generating a so called seek matrix for each compositing area needed (usually just one big one with high resolution is enough). The matrix is a kind of pointer table telling which bins may affect each pixel of the final composite. The indices of radar, elevation, azimuth and range gate of these bins are recorded in this matrix, as well as height, displacement, pixel coverage and constant quality parameters of each bin. Also distances of each pixel from each radar are recorded for distance dependent correction purposes.

The data value representing the wanted CAPPI level or ground level at each pixel is calculated by taking ito account all affecting bin values pointed by the matrix. All kind of quality or correction fields which can be presented in polar coordinates (e.g. beam blocking or attenuation due precipitation) are easy to implement in this method (for example, the system used in FMI applies distance-weighted vertical reflectivity profile correction using time-space averaged profiles and their quality parameters derived for each radar).

Finally, when computing the "optimal" cartesian products the weighting
factors from bins' parameters and their data quality indicators are derived to calculate the optimal data value for each pixel. Several different parallel weighting schemes can be used to make suitable product for different users of the data.

This method also provides simultaneous computing of composites for wanted amount of CAPPI levels. It can also be used to compute e.g. composited echo top and base products and even arbitrary cross sections.

The time consuming computing of seek matrices are done in advance to
achieve compositing times of only few seconds. The only disadvantage is considerably large amount of memory required by matrix and data. Typical size of the matrix for seven radar composite of 1 km resolution is 200 megabytes. However, the method is most adequate for parallel computing, because the composite can be constructed from as small as wanted sub-composites with their own matrices.

Of course "classical" computing using cartesian 3D-grid is also possible,
but pre-processing (interpolation etc) has to be done every time new data arrives, and the grid resolution must be very fine to avoid data loss. Also the quality and correction fields has to be transformed from their original polar form to this cartesian grid. All this can lead to unacceptable long pre-processing times and huge amounts of memory needed to store both data and quality fields in 3D-grid. The advantage using this method is the fastness of production of different radar products from the data in already cartesian grid. Nevertheless, this method seems to be suitableonly for supercomputers these days, but may be useful in the future if capasity of computers increases at current rate.

Harri Hohti, FMI, January 2004