|
|
A component of the safety net ensuring the safety of air
traffic is Short Term Conflict Alert (STCA) systems, which use
ground-based radar to monitor aircraft and alert air-traffic
controllers to potential airspace conflicts, who can then direct
pilots to take suitable evasive routes. The National Air Traffic Service's
(NATS) STCA system covering the busy London terminal airspace
handles over 3000 aircraft per day and its importance is
highlighted by the fact that it is thought that one of the factors
contributing to the midair collision over the border between
Germany and Switzerland in July 2002 was that parts of the STCA
system in the relevant Swiss control station were switched off for
maintenance. It is crucial to raise alerts for the 5-15 daily over
London genuine potential airspace infractions, true
positives, but important to minimise the number of false
positives so as to avoid crying wolf too often to air traffic
controllers who would tend to lose confidence in the system.
The STCA program is a vastly complicated predictive model with more
than 1500 adjustable parameters, such as `vertical closing rate
threshold', which are currently manually adjusted by skilled NATS
staff using a database of about 170000 historical and recent
encounters. This laborious tuning must be carried out in
response to changes in volume of air traffic, local traffic
operational procedures and changes in the regulatory environment.
|
|
As part of an EPSRC programme on Critical Systems and Data-Driven
Technology, we have used evolutionary algorithms for
multi-objective optimisation to locate the Pareto front---the curve
that describes the optimal trade-off between the true positive and
false positive rates. Multi-objective optimisation acknowledges
that when attempting to maximise or minimise two objectives (for
example, price and performance, or here true positive rate and
false positive rate), there will often be optimal solutions that
are not wholly better or worse than each other; for example,
solution A may have better false positive rate than B, but worse
true positive rate and vice versa. Recent advances in evolutionary
algorithms permit the efficient location of the Pareto front of
solutions which are not wholly worse than any other solutions. The
picture shows solutions marked as red dots on the Pareto front
describing the optimal trade-off between genuine and nuisance
alerts for the London airspace.
|
|
The warning time given to air-traffic controllers of an impending
serious encounter is a third objective that may be optimised
simultaneously with the genuine and nuisance alert rates. The
resulting Pareto surface describing the trade-off between the three
objectives is visualised in the picture below by colouring
solutions on the two-dimensional front according to the warning
time.
|
|
Questions such as: "how confident about the optimal rates can we be?" or
"what if, different or equivalent, data were available for training?" can
be answered by statistical bootstrapping methods that quantify the
spread around the front that can be expected by chance.
Discovering the Pareto front took about 12 days of computer time in which
an unmodified STCA system was run in Exeter under the control of an
optimiser, but it was unattended computer time in contrast to the
laborious, skilled and expensive optimisation process undertaken by NATS.
Importantly, the use of an unmodified STCA system means that safety-case
arguments made for the original STCA system can be carried, unmodified,
across to the optimised system: no structural changes to the STCA program
have been made and the parameter values located could, in theory at least,
have been found manually. The STCA system is a single example of a
safety-related system with many parameters that is manually tuned
to changing conditions and the Exeter group is investigating the wider
application of these methodologies to safety-critical systems.
More details in R.M. Everson and J.E. Fieldsend, Multi-objective optimisation of safety
related systems. IEEE Transactions on Evolutionary Computation,
2004. (Under review.)
|