Introduction

 To measure the market risk of a portfolio of traded assets, many banks are increasingly employing internal models based on a methodology called Value-at-Risk (VaR). In an important regulatory innovation [1], the Basle Committee on Banking Supervision has proposed in 1996 that such models may be used in the determination of the capital requirements that banks must fulfill to back their trading activities. In addition, the Bank for International Settlements - Fisher Report [2] urged financial intermediaries to publicly disclose Values-at-Risk. The Group of thirty [3] and ISDA also advocate the VaR concept as a general measure of market risk.

The motivation behind the use of these models is to quantify risk for internal decision making and capital adequacy purposes. The quantification is in terms of the upper limit for loss inherent to a portfolio position with a given pre-specified probability, called level of confidence, and over a fixed period of time in the future holding period. An in depth description of the theory of VaR models is given in [4], [5], [6] and [7].

VaR models have to deal with 4 mathematical components:

1.

Models - for predicting conditional distributions of returns for the underlying risk factors.

2.

Stripping - of instruments so that they are expressed in terms of their underlying risk factors.

3.

Value function of the portfolio - which may be non-linear with respect to the returns of the underlying.

4.

Construction of profit and loss forecast distribution - based on the value function and class of conditional distribution of risk factors.

VaR models may differ from each other in different ways with respect to one or more of the above components. Components 3 and 4 are often mathematically interlinked - and together may be broadly addressed with the following popular methodologies:

• Variance-Covariance - or $\Delta$ approach.
• The $\Delta-\Gamma$ approach.
• Monte-Carlo simulation.
• Scenario simulation.

One of the objectives of this paper is to compare different models for predicting market risk. For this purpose we have chosen a candidate portfolio of spot foreign exchange and metal rates. Since the candidate portfolio is linear - the Variance-Covariance approach for the portfolio function is more than adequate for the purpose of this paper.

There is a considerable amount of work related to VaR models from academics, practitioners and regulators. Traditionally, a large volume of work from academics has been in the field of stochastic modeling of financial time series. This huge volume of work - review article [8] containing 230 references - while being extremely relevant is not directly presented in the context of risk management. Academic work directly addressing the field of risk management is less common but includes a broad range of topics of practical and theoretical relevance such as the formal properties and deficiencies of the VaR concept [9], the problem of actual versus risk neutral distributions [10], the discrepancies between the diversity of implementations of the same model - system risk [11], the determination of capital requirements with internal models [12], the appropriateness of VaR models as a management tool [13] and techniques for forecasting variances and covariances [14,15,16].

Important contributions to VaR models from practitioners include the breakthrough work of J. P. Morgan RiskMetrics giving a complete description of VaR models, criticisms, assumptions and methods [4], methods to judge the VaR model forecast quality [17], concepts that analyze the risk exposure of a portfolio by measuring the risk of the basic blocks of the total portfolio [18] and the probabilistic foundation of scenario simulation models [19].

The contributions by regulators in this field primarily focus on issues related to the integration of VaR models within BIS recommendations and the attendant problems faced in practical implementation of these recommendations. Consequently work by regulators focus on backtesting [20,21] and on issues that relate VaR model performance to the BIS parameters [22], [23], [24] and [25].

This paper covers a cross section of issues focused on by academics, practitioners and regulators. From the academic perspective, we offer some innovations in stochastic modeling of financial time series - and offer a stringent testing framework for new models. From the regulators perspective, we introduce and discuss various measures for evaluating model performance in the context of risk management. From the practitioners perspective we carry out the entire evaluation using real data and try to construct performance measures which strike a balance between risk over-estimation and risk-under estimation.

In sections 2 through 6 of this paper, we evaluate the absolute and relative performance of VaR estimates based on historical simulation, the rectangular moving average, the exponential moving average, and the conditional variance forecasted by the GARCH process. In dealing with this issue we also discuss the meaning of performance in terms of different techniques to evaluate the success of the VaR model. In section 7, we discuss the issue of stochastic error in covariance matrix estimation and the inter-relation of this problem to the following issues:

• Possible singularities in the covariance matrix due to rank defects caused when the sample size is smaller than the dimension of the correlation matrix (which is generally true when using the popular figure of 250 days history).
• Systematic underestimation of risk arising out of the re-optimization of the portfolio (to minimize VaR) based on the estimated covariance matrix.

We end the discussion with some speculation on the innovations necessary to handle this problem. Although section 7 is directly unrelated to the earlier sections - we believe that it is a crucial point which must be emphasized for the completeness of this paper. We end the paper with a summary of innovations and recommendations for better overall handling of risk measurement.