What is Model Risk and What Can I Do About it?

Model risk is the risk that the market models in investment banking do not properly reflect the reality. This risk is often neglected or simply ignored. But, it is one of the most important risks as we could see in the mispricing of CDO, ABS, MBS etc at the beginning of the financial crisis (early works saw this already in 2004 [2]).

Model Risk is Underestimated

From physics, it is clear that a model never captures the full reality. It always describes only parts of the reality. But, every good model has the property to be falsifiable. The currently used models in physics like general relativity theory and quantum mechanics are falsifiable and have been challenged in the past decades: Both theories deliver extremely precise forecasts and have not been falsified.

In finance and financial mathematics, the situation is different. Many theories like CAPM, Markowitz portfolio optimization, Black-Scholes option pricing, chart analysis and others have been developed and all these theories are more or less successful. But, a close look reveals that the model results are always imprecise or even false.

That means, while in physics good theories must be falsifiable, in finance all theories are false. Even the simplest assumptions like: “People prefer more money to less money” are not always true. In some cases people prefer less money e.g. for ethical reasons.

What Can We Do About Model Risk?

This is a though question. First of all, you must know the limitations of your models.

Challenge model assumptions: Liquidity, transactions costs, taxes, counter party risk, operational risk – all this is often ignored modeling [1].
Back-test your model with historical data: Prefer data in back-testing which has not been used in calibration of the input data [1].
Include the parameter estimation in the back-testing: Parameters should be constant in time and the estimation procedure should be statistically significant. E.g. try to simulate a GARCH (1,1) model and estimate the now known parameters from the simulation. You will be surprised how many observations are required for a stable parameter estimation.
Stress your model: See your model performance in crash scenarios, high volatility regimes or extreme correlations of 1 resp. -1.
Monitor your model performance: The model performance has to be measured on a regular basis [1]. Otherwise, you will miss important market shifts, e.g. in correlations.
Use more than one model: Use a portfolio of models and watch the extremes [3,4, 5,7].
Try to benchmark with a model with very few assumptions: In many cases, bounds on the price of an option can be obtained without a limited market model. Static hedging can deliver insight without considering the market dynamics. Simulation-Based Hedging [6] can deliver prices and risks in any sophisticated market model.
Of cause, make sure that your model implementations are numerically sound [1].

And then, you must think of unknown future events. This requires to be creative as I posted in the previous article: Risk Management must be creative.

Conclusion

Finance is a science about people’s behavior and thus not as strict as e.g. physics. All financial models have tight limitations and you have to exercise great care using them. Knowing the limitations of your models will help you maintaining a sustainable business model.