By James L. Glueck, CFA,FRM, SVP, Analytics and Della Zheng, Ph.D., FRM,
Vice President, Analytics from MountainView-McGuire.
Ahead of the Stress Testing USA: CCAR & DFAST Congress, James and Della share with us their insight into a full circle approach to model validation.
James & Della, please each tell the Risk Insights readers more about yourself, your experiences, and you current professional focus.
Our custom model validation team brings accomplished and experienced resources deeply knowledgeable of industry practices, leading quantitative techniques, and regulatory requirements to the model risk management needs of our varied client base. With more than 50 years of combined team industry experience, the MountainView-McGuire team ensures delivery of top-quality model validation service and support serving the validation needs of both DFAST and CCAR institutions.
James Glueck has more than 20 years of industry experience serving in senior roles for both 1st and 2nd line functions within Treasury, balance sheet, model risk management teams, both as internal and external advisory resource. His current professional focus is in the area of econometrics supporting stress testing and loss forecasting as well as the evolving nature of governance requirements supporting the entire model lifecycle.
Dr. Della Zheng brings a wide range of skills and experience in model development, implementation, and validation, covering various areas including asset liability management, stress testing, and forecasting. She is currently focusing on econometric model validation related to stress testing and/or business-as-usual (BAU).
As stress testing USA: CCAR&DFAST Congress, you will be speaking on your insight on balancing conceptual soundness review with independent testing: why do you believe this is relevant and what do you think are the key things to remember?
A full-circle approach to model validation should encompass both qualitative as well as quantitative approaches, consistent with leading practice and current supervisory standards. When validating econometric models, independent affirmation of model specification separate from independent misspecification testing is a critical component of the model review process. And when it comes to assessing for possible model misspecification during the development process, over-testing can be useful, also known as the test, test, test (TTT) approach to model validation.
Can you give a brief overview of the TTT approach to independent model review and validation? In your experience, what are the key considerations that need to be made?
When applying the test, test, test (TTT) approach to independent model review and validation, it is important to distinguish between model specification and misspecification testing. Also referred to by practitioners as “testing down,” when combined with the use of model diagnostics, TTT facilitates the identification of potential model specification inadequacies. In contrast, specification testing aims to evaluate the validity of an alternative model specification. The TTT approach is consistent with a full-circle econometric view of model validation practice, separate from but complementary to more qualitative-based approaches for assessing model conceptual soundness.
Why is it essential that financial institutions undertake a full circle approach to model validation? What are the limitations?
A full-circle approach to model validation is essential for financial institutions because it is the approach required by current supervisory guidance. Beyond strict compliance, a full-circle model validation approach empowers key stakeholders in the model lifecycle process and ensures sound model development, implementation, and use.
What are the unique qualitative & quantitative considerations, approaches, and limitations?
Full-circle model validation approaches as presented here require expertise in all areas of econometric model estimation, deployment, and use. Larger financial institutions typically staff such resources internally; smaller shops combine internal resources with external knowledge and expertise. Regardless of size or complexity, most financial institutions can benefit from the targeted deployment of independent validation resources for any or all components of full circle model review, including both qualitative and quantitative assessments as well as a full battery of specification, misspecification, and diagnostic testing.
What does the future hold for stress testing professionals and how can they keep up with the increasing changes in the industry?
Data quality, data granularity, and data access will remain key challenges for stress testing model development and calibration. Additionally, model performance measurement and monitoring will be critical particularly with respect to model updates applied annually consistent with regulatory reporting cycles and requirements. Finally, anticipating and incorporating shifting regulatory expectations, standards, and requirements across the lifecycle of stress testing models will be key for financial institutions of all sizes and complexity levels.
