Sep 19, 2012

Model Validation: The ‘Holy Grail’ for Risk Mitigation?

From risk mitigation to quality assurance and model performance…model validation means many different things to many different people these days. Given today’s regulatory climate, we are seeing a growing trend in financial institutions that are investing in independent resources to ensure that their models have been implemented properly?and that they are performing as expected. In addition to providing good hedges in all market conditions, market practitioners want to see that each model has realistic behavior; and, they want to understand model behavior limitations under extreme market scenarios. Really, when we think about it, who isn’t seeking a little additional ‘validation’ these days?

So, what are some of the most popular kinds of model validation tests? Comparison to Market Prices is always the best test, but is not always available (as we all have learned by now). When market prices are not an option, we must consider other options, such as generic tests, which can be applied across all models?or we can use other tests that are instrument or model-specific. There are tests that examine the numerical implementation, to see whether it is true to the underlying mathematics, while others test the fidelity of the model dynamics to market behavior, to see whether the model when correctly implemented, hedges instruments well, explains David Eliezer, Vice President at Numerix, during a recent webinar discussion: “Pricing & Valuations: Mitigating Risk.”

Model Validation 101: What Are Some of the Basics Market Practitioners Need to Know?

When it comes down to model validation and standardized testing, there are standardized tests for mathematical or financial correctness, some are instrument and model specific and some are considered ‘generic’ tests.

Let’s have a closer look:

I.   Standardized tests for Mathematical Correctness include some of the following:

           Instrument and Model-Specific Tests:

  • Parallel Reimplementation;
  • Limiting Cases or Identities for Payoff;
    (e.g. Knock-in + Knock-out = Vanilla, Barrier > Vanilla as Barrier >  infinity);
  • Limiting Cases or Identities for Model;
    (e.g. Heston > Black-Scholes, Romano-Touzi Mixing Theorem); or
  • Value with a collection of different models to show that the prices are similar
    Model and Instrument Independent, ‘Generic’ Tests include some of the following:
  • Delta-Vega Smoothness Tests
    (Is there a “jitter” or oscillation mixed in with the pricing that throws off greeks even if model price is accurate?);
  • Calibration Round Trip Tests
    (Does our calibration method re-price the calibrated instruments accurately?  When a calibration problem is set up with a known solution, can the calibrator find it?);
  • Convergence Tests
    (Does our pricing converge, and is the production price close to the converged value?)

II.    Additionally, Model Validation tests for Financial Correctness test for the performance of the model's hedges and can include the following:

  • Calibration Stability Tests
    (Does your model have similar calibration parameters from day-to-day?  Does your model price the same vanilla similarly from day-to-day?);
  • Calibration Error Tests
    (Does your model, when calibrated to vanillas from the market, price all the calibration instruments accurately?);
  • PnL Attribution Tests
    (Does our model account for changes in PnL of the instrument due to changes in market data well without calibration?  If we use this model to hedge, will our hedges hedge our changes in value? – This is similar to the Calibration Stability Test)
  • Variance of Hedged Portfolio Test
    (What is the variance of the hedged portfolio -- similar to PnL Attribution Test)?
  • Cost of Hedging Test
    (Does the initially calculated price accurately reflect the eventual cost of hedging?)

Given some of the extreme market behaviors we’ve all witnessed in the past few years, and the ever-evolving regulatory landscape we are all facing in 2012 and beyond, it is no wonder that more and more folks are seeking ways to validate their models for pricing and valuation and to mitigate their risk.

For more information on model validation, contact, or access a recorded version of the risk mitigation webinar now.

Need Assistance?

Want More From Numerix? Subscribe to our mailing list to stay current on what we're doing and thinking

Want More from Numerix?

Subscribe to our mailing list to stay current on what we're doing and thinking at Numerix

Subscribe Today!