Next: Measure 1
Up: Discussion and Results
Previous: Measures sensitive to number
To capture the degree of exceedence - we have presented performance measures
4 and 5. The use of the mean log-likelihood of exceedent or large events
(measures 4 and 5 respectively) in the out of sample is a sophisticated measure.
For the conditional Gaussian distributions used in models 2 through 5, the probability
density is strictly monotonically decreasing on both sides of the zero mean - and we are thus
guaranteed that as desired
for each exceedent event becomes
increasingly negative as the degree of exceedence of xt+1 increases.
In the case of historical simulation however (model 1), this is not strictly true
- since pt can be and is most likely multi-modal.
If according to the model - a large event is more likely than a smaller event of
lower size (as in the case of a multi-modal distribution) - and if the
out-of-sample data confirms this distribution - then this is a truth which is
captured by the model and the model should not be penalized for such an occurrence.
On the other hand, if the conditional distribution is multi-modal, but the realizations
do not fit this distribution - then the model should be penalized.
This is exactly what is achieved by the mean log-likelihood measures 4 and 5 - by
expressing exceedence as a function of the probability density of the event
rather than simply looking at the size of the exceedence.
In general, for large populations, the mean log-likelihood maximizes when the
predicted and realized distributions of an event class match up.
Next: Measure 1
Up: Discussion and Results
Previous: Measures sensitive to number