next up previous
Next: Measure 1 Up: Discussion and Results Previous: Measures sensitive to number

Measures sensitive to degree of exceedence

To capture the degree of exceedence - we have presented performance measures 4 and 5. The use of the mean log-likelihood of exceedent or large events (measures 4 and 5 respectively) in the out of sample is a sophisticated measure. For the conditional Gaussian distributions used in models 2 through 5, the probability density is strictly monotonically decreasing on both sides of the zero mean - and we are thus guaranteed that as desired $\ln(p_t(x_{t+1}))$ for each exceedent event becomes increasingly negative as the degree of exceedence of xt+1 increases. In the case of historical simulation however (model 1), this is not strictly true - since pt can be and is most likely multi-modal. If according to the model - a large event is more likely than a smaller event of lower size (as in the case of a multi-modal distribution) - and if the out-of-sample data confirms this distribution - then this is a truth which is captured by the model and the model should not be penalized for such an occurrence. On the other hand, if the conditional distribution is multi-modal, but the realizations do not fit this distribution - then the model should be penalized. This is exactly what is achieved by the mean log-likelihood measures 4 and 5 - by expressing exceedence as a function of the probability density of the event rather than simply looking at the size of the exceedence. In general, for large populations, the mean log-likelihood maximizes when the predicted and realized distributions of an event class match up.


next up previous
Next: Measure 1 Up: Discussion and Results Previous: Measures sensitive to number