Addressing the challenges of validating Stress Testing Models
Written interview with Philipp Andres from Fintegral
CFP interviews Philip Andres, Manager at Fintegral for insights on the key challenges of vaildating stress testing models. Philip will be providing a practical case study at CFP’s upcoming Stress Testing Europe 2015 Congress taking place later this month.
Q1) Philipp, please tell us a bit more about your experience, expertise and background, as well as the financial institutions and clients you serve?
Fintegral has been involved in stress testing since 2010. We have built, validated and run stress testing models across a range of risk types, and managed complex stress tests in their entirety. Currently, we are working on CCAR models for two major international banks.
Personally, my current work involves developing a pricing engine for exotic FX derivatives to quantify credit value adjustment (CVA) charges and counterparty credit limits for a major bank. The challenge is to calibrate this engine to deliver accurate, future derivative valuations in benign and stressed periods. These valuations enable our client to determine his credit counterparty risk on a netting set level and device strategies to mitigate these risks today.
Our team consists of stress testing, FX pricing and CVA specialists which bring deep knowledge to the project. This mixture of competencies ensures our client is able to deliver within his internal and external deadlines. For example, we deliver the model documentation for the internal model validation and prepared the presentation to explain the approach to the FED. This is to ensure that our client passes his upcoming regulatory examination. The models in question feed into the bank’s CCAR stress test, and have to be shown to work properly under stress.
My background is in statistics and economics, in which I graduated from Cambridge with a PhD. Working for a small and fast growing firm exposed me to a broad spectrum of exciting stress testing and other risk modelling projects. These included modelling LGDs in the commercial real estate and financial sector and engineering sector and name concentration risk engines from the ground up.
Most of our clients are based in the city of London or in Switzerland, and include top tier universal and investment banks. Fintegral employees are recognised expert across a wide range of risk types, including credit risk and model risk.
Q2) You will be providing a practical case study at Stress Testing Europe 2015. What are the critical components and challenges of modelling the spill-over effects of cross country exposures
National regulators such as the Bank of England provide financial institutions with details on the expected evolution of key macroeconomic variables during a hypothetical deterioration of the economic climate. Some, but not all, regulators also prescribe the trajectory of foreign macroeconomic variables. In a first step, the national scenarios are applied by banks to their domestic exposures in their trading and banking books.
Taking a British perspective, the question arises as to which macroeconomic spill-over effects such a national stress scenario implies on individual continental European economies. These knock-on effects in the macroeconomic variables in turn affect the credit-worthiness of the bank’s European exposure in the same way the adverse conditions at home impacted the quality of the domestic exposure.
The case study will demonstrate how to quantify changes in growth, unemployment or interest rates in any European country in response to the British shock. To this end, we adopted a Global Vector Autocorrelation (GVAR) model, which builds on a well-established academic literature (see e.g. Pesaran, Schuermann & Weiner (2004)) on macroeconomic modelling and applies these techniques to quantify the macroeconomic spill-over effects.
Q3) At CFP’s 3rd Annual Stress Testing Europe Congress, you will be presenting on the challenges of Validating Stress Testing Models. In short, what would you say are the key and critical challenges in this area at the moment?
In my experience, problems with data integration across different IT systems and business lines are some of the biggest challenges facing stress testing professionals. Despite considerable efforts to consolidate data sources and large investments across IT systems, their remains scope for further improvements.
The most common challenge is how to integrate stress testing results from decentralized business units into one common set of consistent results, which can then be put forward to the regulator. Some units may rely on legacy systems, adding another layer of complexity of ensuring the bank’s stress test results are based on a coherent framework. Other units may employ different IT systems, booking systems or modelling frameworks, which have often been created independently for the specific needs of that particular group. The same applies to units that were (recently) taken over.
Stress testing requires the same scenarios to be applied across the whole organisation. But consistency is not a given when different groups using differing systems are asked to perform the same exercise. Keeping track of the data flow and the data manipulation that the different business units apply is also a serious challenge.
However, stress testing exercises help financial institutions to foster their system integration and develop more efficient back-office functions, thus realising synergies and saving costs in the long run.
A second key challenge relates to calibrating the banking and trading book risk models to stress scenarios. In the banking book, consider a typical probability of default (PD) model with macroeconomic explanatories, which has been over-fitted using observations from benign periods (TTC calibration). Models like these will struggle to cope with stressed inputs.
Challenging calibration problems are also encountered in the trading book, as translating (stressed) macroeconomic variables into calibration parameters of the models used to price derivatives is not always straightforward. For example, while commodity prices in general might be linked to the evolution of macroeconomic growth, different types of oil (Brent vs WTI) will react differently. In addition, under certain stress scenarios, interest rates are expected to turn negative. Standard interest rate models however assume log-normally distributed rates. Reconciling the macroeconomic stress scenarios with bank internal, arbitrage free, pricing models is at times infeasible or requires significant investment of time and resources.
In my experience, dealing with stressed CVA requires re-modelling the value of the collateral during future market turbulences to reduce the net exposure. This, together with the high computational effort of computing future valuations requires a powerful and robust IT infrastructure. Looking at more exotic derivatives, it also becomes necessary to consider how the instruments could be hedged during a crisis to mitigate potential losses from defaulting counterparties, which requires deep product specific knowledge.
Last but not least, the formal requirements regulators demand have increased considerably during the last year or so. Today, comprehensive and well written documentation in line with the formal expectations of regulators needs to be high on the model owner’s agenda before facing the regulator.
To summarize, data integration and aggregation as well as generating comprehensive and consistent stress scenarios in line with the regulatory specifications remain the key challenges stress testing professionals face today.
Q4) Looking ahead how do you see the role of the Stress Testing professionals change?
Looking ahead, I think stress testing will become even more of a standard discipline in the financial services industry. Even smaller houses and hedge funds increasingly see the need to conduct their own stress tests. This is because the scenario inputs and stress test outcomes are more intuitive than e.g. VaR measures, and hence easier to discuss on a board level. Approaches to stress testing which permit the VaR and scenario approaches to be effectively combined are likely grow in popularity.
However, having invested much time and effort into developing stress testing engines, banks should make better use of this powerful tool. Conducting stress tests as a purely regulatory exercise is a missed opportunity. Instead, the stress tests can be used to identify adverse as well as positive business scenarios to study their effect on individual segments of the business. This establishes how margins in different markets will develop under different scenarios to identify new opportunities and allows to test the earning resilience of different units. Equipped with this knowledge, senior management will be better able to make informed investment decisions. The ideal planning tool is a stress testing engine which permits the impact of scenarios to be computed, at least to first order, close to real time.
At the same time, regulators will continue to face pressure to avert the next crisis, thus producing tougher scenarios at more frequent intervals. To keep up with these moving goalposts and the increased requirements for stringent documentation and for a better understanding of the model behaviour, banks need to invest in improved model governance tools and IT infrastructures.
