Harri Hohti, FMI:
Suboptimal compositing algorithms 

The compositing of radar network data has traditionally been done 
by combining PPI or CAPPI products of individual radars. Several different 
principles of choosing the data, like nearest radar, closest to earth, 
maximum, average and different combinations of these have been used in 
compositing algorithms with varying success. 

Totally new problems will appear if something else than horizontal 
products, e.g. composited cross sections are needed, but those 
are left out of this study. 

Even in case of optimally corrected cartesian products, one problem remains: 
How to choose between two or more geometrically equal, but different 
reflectivity values, if quality of these values are not known? 
An inevitable consequence of this is that there must be lots of cases 
where quality parameters dominates the data selection between two or 
more radars, not the geometry. 


Problems related to cartesian products 

The "factory made" cartesian products themselves have some buildt-in problems  which causes information loss or bias: 

- Near the radar the coordinate conversion itself causes the loss of 
  data due to cartesian resolution being coarser than the the polar one. 
  This is true with PPI and CAPPI products both. 

- The method used to interpolate data between elevation angles affects 
  the data when CAPPIs are used as the cartesian products. This causes 
  a kind of inversion problem; it isn't known if the data value in 
  some pixel is the actual measured one or some interpolated/extrapolated 
  one. So the reliability of the data has no measure. 

- Quality indicators and correction terms derived from effects of 
  interpolation, attenuation, beam blocking etc are perhaps taken 
  into account when the cartesian product is constructed inside the 
  radar software, but these can't be extracted from the product itself, 
  or the data format does not allow inclusion of these. 

Conclusion: quality indicator fields are necessary when selecting/ 
            calculating the optimal values of data from two or more 
            radars. 



Problems related to selection rules 


Even having the best possible cartesian products available there 
is still problems with each of compositing principles. 
In the following list it is assumed that optimal estimation of 
ground precipitation is the goal of compositing. 

To use these rules there must at least information of elevation 
angles and radar heights available, so that the height of measurement 
at each point can be calculated. It is also assumed that calibration 
of radars are equal, which is usually not true. 

- closest to earth:  The pixel closest to earth don't necessarily 
                     contain the most qualified data. Especially 
                     clutter and beam blocking can easily contaminate 
                     the data from lowest elevation angle, and attenuation 
                     due precipitation can cause underestimation of 
                     reflectivity if not corrected. 

- nearest radar:     Works correctly only when scanning strategies of 
                     radars are equal and radars are situated at the same 
                     height. 

- maximum:           This rule has several problems. It works considerably 
                     fine only if some of the radars in network suffers 
                     serious beam blocking. Otherwise a bad choice because 
                     the vertical profile of reflectivity is not constant. 
                     Overhanging precipitation is the worst problem when 
                     using this rule, and bright band can also cause 
                     overestimation of ground precipitation. 

- average:           Usually the worst choice, because output values 
                     are always biased. Especially in case of small but 
                     intesive convection cells both distance dependent 
                     beam height and difference in resolution causes 
                     smoothing and leads to serious underestimation of 
                     precipitation. 


Requirements due the users of data 
---------------------------------- 

It is also important to realize that composite made by using 
one, even good algorithm may not necessarily fulfil the needs 
of the end-user of the data. Even the "ground truth" product 
can have totally different requirements. For example, for hydrological 
purposes there is no need for nice pictures, only as correct as possible 
data is important. On the other hand, a radar composite shown on 
commercial TV-channel has almost opposite requirements;  the picture must 
be just fine looking, nice and smooth, without any need for "correct" 
values of single pixels. 

So there should be a possibility for fast production of composites which 
obeys several different requirements. This means that design, tuning and 
implementation of new algorithms or selection rules should be flexible. 

Possible solutions 

One possible solution to the problem developed at FMI uses polar volumes 
directly. It is based on generating a so called seek matrix for each 
compositing area needed (usually just one big one with high resolution is 
enough). The matrix is a kind of pointer table telling which bins may affect 
each pixel of the final composite. The indices of radar, elevation, 
azimuth and range gate of these bins are recorded in this matrix, as well 
as height, displacement, pixel coverage and constant quality parameters 
of each bin. Also distances of each pixel from each radar are recorded 
for distance dependent correction purposes. 

The data value representing the wanted CAPPI level or ground level 
at each pixel is calculated by taking ito account all affecting bin 
values pointed by the matrix. All kind of quality or correction fields 
which can be presented in polar coordinates (e.g. beam blocking or 
attenuation due precipitation) are easy to implement in this method 
(for example, the system used in FMI applies distance-weighted 
vertical reflectivity profile correction using time-space averaged 
profiles and their quality parameters derived for each radar). 

Finally, when computing the "optimal" cartesian products the weighting 
factors from bins' parameters and their data quality indicators are 
derived to calculate the optimal data value for each pixel. 
Several different parallel weighting schemes can be used to make 
suitable product for different users of the data. 

This method also provides simultaneous computing of composites for 
wanted amount of CAPPI levels. It can also be used to compute 
e.g. composited echo top and base products and even arbitrary cross 
sections. 

The time consuming computing of seek matrices are done in advance to 
achieve compositing times of only few seconds. The only disadvantage is 
considerably large amount of memory required by matrix and data. 
Typical size of the matrix for seven radar composite of 1 km resolution 
is 200 megabytes. However, the method is most adequate for parallel 
computing, because the composite can be constructed from as small as 
wanted sub-composites with their own matrices. 

Of course "classical" computing using cartesian 3D-grid is also possible, 
but pre-processing (interpolation etc) has to be done every time 
new data arrives, and the grid resolution must be very fine to avoid 
data loss. Also the quality and correction fields has to be transformed from 
their original polar form to this cartesian grid. All this can lead to 
unacceptable long pre-processing times and huge amounts of memory needed 
to store both data and quality fields in 3D-grid. The advantage using this 
method is the fastness of production of different radar products from the 
data in already cartesian grid. Nevertheless, this method seems to be 
suitable only for supercomputers these days, but may be useful in the 
future if capasity of computers increases at current rate. 

Harri Hohti, FMI, January 2004