Harri Hohti, FMI:
Suboptimal compositing algorithms
The compositing of radar network data has traditionally been done
by combining PPI or CAPPI products of individual radars. Several different
principles of choosing the data, like nearest radar, closest to earth,
maximum, average and different combinations of these have been used in
compositing algorithms with varying success.
Totally new problems will appear if something else than horizontal
products, e.g. composited cross sections are needed, but those
are left out of this study.
Even in case of optimally corrected cartesian products, one problem remains:
How to choose between two or more geometrically equal, but different
reflectivity values, if quality of these values are not known?
An inevitable consequence of this is that there must be lots of cases
where quality parameters dominates the data selection between two or
more radars, not the geometry.
Problems related to cartesian products
The "factory made" cartesian products themselves have some buildt-in problems which causes information loss or bias:
- Near the radar the coordinate conversion itself causes the loss of
data due to cartesian resolution being coarser than the the polar one.
This is true with PPI and CAPPI products both.
- The method used to interpolate data between elevation angles affects
the data when CAPPIs are used as the cartesian products. This causes
a kind of inversion problem; it isn't known if the data value in
some pixel is the actual measured one or some interpolated/extrapolated
one. So the reliability of the data has no measure.
- Quality indicators and correction terms derived from effects of
interpolation, attenuation, beam blocking etc are perhaps taken
into account when the cartesian product is constructed inside the
radar software, but these can't be extracted from the product itself,
or the data format does not allow inclusion of these.
Conclusion: quality indicator fields are necessary when selecting/
calculating the optimal values of data from two or more
radars.
Problems related to selection rules
Even having the best possible cartesian products available there
is still problems with each of compositing principles.
In the following list it is assumed that optimal estimation of
ground precipitation is the goal of compositing.
To use these rules there must at least information of elevation
angles and radar heights available, so that the height of measurement
at each point can be calculated. It is also assumed that calibration
of radars are equal, which is usually not true.
- closest to earth: The pixel closest to earth don't necessarily
contain the most qualified data. Especially
clutter and beam blocking can easily contaminate
the data from lowest elevation angle, and attenuation
due precipitation can cause underestimation of
reflectivity if not corrected.
- nearest radar: Works correctly only when scanning strategies of
radars are equal and radars are situated at the same
height.
- maximum: This rule has several problems. It works considerably
fine only if some of the radars in network suffers
serious beam blocking. Otherwise a bad choice because
the vertical profile of reflectivity is not constant.
Overhanging precipitation is the worst problem when
using this rule, and bright band can also cause
overestimation of ground precipitation.
- average: Usually the worst choice, because output values
are always biased. Especially in case of small but
intesive convection cells both distance dependent
beam height and difference in resolution causes
smoothing and leads to serious underestimation of
precipitation.
Requirements due the users of data
----------------------------------
It is also important to realize that composite made by using
one, even good algorithm may not necessarily fulfil the needs
of the end-user of the data. Even the "ground truth" product
can have totally different requirements. For example, for hydrological
purposes there is no need for nice pictures, only as correct as possible
data is important. On the other hand, a radar composite shown on
commercial TV-channel has almost opposite requirements; the picture must
be just fine looking, nice and smooth, without any need for "correct"
values of single pixels.
So there should be a possibility for fast production of composites which
obeys several different requirements. This means that design, tuning and
implementation of new algorithms or selection rules should be flexible.
Possible solutions
One possible solution to the problem developed at FMI uses polar volumes
directly. It is based on generating a so called seek matrix for each
compositing area needed (usually just one big one with high resolution is
enough). The matrix is a kind of pointer table telling which bins may affect
each pixel of the final composite. The indices of radar, elevation,
azimuth and range gate of these bins are recorded in this matrix, as well
as height, displacement, pixel coverage and constant quality parameters
of each bin. Also distances of each pixel from each radar are recorded
for distance dependent correction purposes.
The data value representing the wanted CAPPI level or ground level
at each pixel is calculated by taking ito account all affecting bin
values pointed by the matrix. All kind of quality or correction fields
which can be presented in polar coordinates (e.g. beam blocking or
attenuation due precipitation) are easy to implement in this method
(for example, the system used in FMI applies distance-weighted
vertical reflectivity profile correction using time-space averaged
profiles and their quality parameters derived for each radar).
Finally, when computing the "optimal" cartesian products the weighting
factors from bins' parameters and their data quality indicators are
derived to calculate the optimal data value for each pixel.
Several different parallel weighting schemes can be used to make
suitable product for different users of the data.
This method also provides simultaneous computing of composites for
wanted amount of CAPPI levels. It can also be used to compute
e.g. composited echo top and base products and even arbitrary cross
sections.
The time consuming computing of seek matrices are done in advance to
achieve compositing times of only few seconds. The only disadvantage is
considerably large amount of memory required by matrix and data.
Typical size of the matrix for seven radar composite of 1 km resolution
is 200 megabytes. However, the method is most adequate for parallel
computing, because the composite can be constructed from as small as
wanted sub-composites with their own matrices.
Of course "classical" computing using cartesian 3D-grid is also possible,
but pre-processing (interpolation etc) has to be done every time
new data arrives, and the grid resolution must be very fine to avoid
data loss. Also the quality and correction fields has to be transformed from
their original polar form to this cartesian grid. All this can lead to
unacceptable long pre-processing times and huge amounts of memory needed
to store both data and quality fields in 3D-grid. The advantage using this
method is the fastness of production of different radar products from the
data in already cartesian grid. Nevertheless, this method seems to be
suitable only for supercomputers these days, but may be useful in the
future if capasity of computers increases at current rate.
Harri Hohti, FMI, January 2004