Machine Learning and Model Risk Management

The Quintessential Technology Source for Corporate Financial Professionals

Machine Learning and Model Risk Management

Peter Quell, Head of Portfolio Analytics for Market and Credit Risk, DZ BANK

Machine learning has permeated almost all areas in which inferences are drawn from data. The range of applications in the financial industry spans from credit rating, loan approval processes in credit risk to automated trading, portfolio optimization and scenario generation for market risk. Machine learning techniques can also be found in fraud prevention, anti-money laundering, efficiency / cost control and marketing models. Machine learning has demonstrated significant uplift in these business areas, and the use of machine learning will continue to be explored in the financial industry.

The banking industry is becoming increasingly aware of model risks related to the use of machine learning techniques for risk management purposes. Even though quite comprehensive, regulatory guidance such as the Fed’s SR 11-7 will not answer all financial practitioners’ questions related to the implementation and use of machine learning algorithms in their daily business.

What are the main challenges when it comes to the application of machine learning in a regulatory context?

Explainability / Interpretability.  One should be in a position to explain how the algorithm makes a prediction or decision for one specific case at a time.

Overfitting. One should recognize that there is some amount of randomness in the training data. If not taken care of, algorithms show good performance on training data – but fail on data not seen before.

Robustness and transient environments: One should account for the fact that markets or environments can change, that calls for a good balance of adaptability and robustness.

Bias and adversarial attacks: Compared to classical statistics there is a much more prominent role for (training) data in machine learning applications.

Of course, some of these issues have been addressed within the machine learning community. What is needed now is the transfer to the banking industry without “reinventing the wheel”. For that  reason the Model Risk Managers’ International Association (mrmia.org) issued a white paper to discuss some (banking) industry best practices. That would only be a starting point since the applications are rapidly evolving.

How should the Model Risk Governance react to the challenges mentioned above?

Model review: If machine learning algorithms frequently change their “inner workings”, how should model validation react? What should be the contents of the validation activity? How should aspects of conceptual soundness (Fed’s SR 11-7) be treated?

Model development, implementation and use: How to account for the more prominent role of data? What level of complexity can users handle? What kind of explanations would be accepted by users or by senior management?

Model identification and registration: How to account for model complexity, role of data, model recalibration within the model inventory?

Excellent quality standards: Existing frameworks need to be enhanced by additional checks for overfitting and sensitivity analysis to test for robustness. Tests for possible bias and discrimination may be reviewed with respect to reputational risk.

Some banks have already developed frameworks to deal with model risks of machine learning applications, while other banks are still in the midst of soul searching for viable starting points. There definitely is a need to share emerging industry best practices and to develop a comprehensive framework to assess model risks in machine learning applications. We invite all risk professionals to share their views on model risk and machine learning under aimrm@mrmia.org.

Weekly Brief

Read Also

Digital Trends in Payments

Digital Trends in Payments

Kaloyan Yanchev, Head of Payment Methods at Nexo
Don't leave your fintech business exposed to threat

Don't leave your fintech business exposed to threat

Nick Rugg, Senior Underwriter of Fintech & Financial Institutions, Markel International
Empathy Matters More Than Your Technology Budget

Empathy Matters More Than Your Technology Budget

Kelly Dakin, Chief Digital and Customer Experience Officer at Atlantic Union Bank
Strategy as Pasta Puttanesca Prepared by the CFO

Strategy as Pasta Puttanesca Prepared by the CFO

Henrik Tham, Head of Strategy & Transformation, Large Corporate & Institutions, Swedbank
Should They Stay (at home) or Should They Go (back to the office) Now?

Should They Stay (at home) or Should They Go (back to the office) Now?

Dr. Achim Wolter, Head People and Organisational Development, Baloise Group, Switzerland
The Customer Is At The Heart Of Everything We Do?

The Customer Is At The Heart Of Everything We Do?

Mathew Pearce, Global Head of Performance & Strategy, GB & M Finance, HSBC