Validating AI & Machine Learning Models — Lessons Learned from the Banking Industry

Background on AI and ML

What is Model Risk?

  • A model may have fundamental errors and produce inaccurate outputs when viewed against its design objective.
  • A model may be used incorrectly or inappropriately, or there may be misunderstandings about its limitations.
  • Is the data good?
  • Where did you get it?
  • Do you have accurate representations?
  • Was the model implemented appropriately?
  • What are your test methodologies?
  • The first line: builds models and provides documentation
  • The second line: checks those against the first line

Seven Lessons from the Banking Industry

#1 The Need for Default Best Practices

#2 Standardization Reduces the Labor of Documentation

#3 Process Reduces Variability

#4 Validating AI and Machine Learning Models with a Risk-Based System is Necessary

#5 Automation Makes Benchmarking Easy

#6 Transparency Means Different Things to Different People

#7 Automation Reduces Implementation Risk

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
ODSC - Open Data Science

ODSC - Open Data Science

94K Followers

Our passion is bringing thousands of the best and brightest data scientists together under one roof for an incredible learning and networking experience.