Maybe it’s my “availability bias,” but AI seemed to be the theme of last week’s Auto Finance Summit. There was one dedicated session while others, like residual risk and subprime credit, had AI in the background.
The exhibit hall featured the usual AI-based businesses: Upstart, Zest, SAS, et al. In today’s post, I’ll summarize what I learned from the conference.
One panelist framed AI as “the same thing we’ve done with credit scores for the last thirty years.” While technically incorrect – no one would describe a scorecard as AI – this framing has merit. As with credit scoring, any AI model must be monitored for “drift,” and continually retrained.
Welcome to ML Ops
This brings us to MLOps or, as Experian calls it, Model Ops – and it’s not easy. Experian reports that 55% of models never make it into production. Their survey, which I have been excerpting on my Twitter feed, is filled with stats like this.
MLOps is like Dev Ops, only you have to version the data as well, and the code is guaranteed to rust.
Here is how I described MLOps to an engineering manager: Think of the work your team does to control code and manage a pipeline. MLOps is like that, only you have to version the data as well, and the code is guaranteed to rust.
There are good no-code AI tools, like Google AutoML, which I wrote about here, SageMaker, and SAS Viya. As an old-school Python coder, I was gratified to see that these tools are in the minority.
Not Just for Credit Scoring Anymore
The “same as credit scores” framing is instructive in another way, too. This is where lenders first learned the power of predictive analytics, and began to build a capability that now includes targeted marketing, fraud detection, behavioral scoring, and more.
While generative AI is an exciting and rapidly advancing technology, the other applications of AI … continue to account for the majority of the overall potential value – McKinsey
There was a “gotcha” moment in one session, where the panelists had to admit they’re not doing anything with Generative AI. But, why would they? There are at least a dozen low-hanging use cases for AI classifiers and regressors, as I mentioned in What Is Real AI?
The AI/ML Development Journey
The most practical advice came from CUDL President Brian Hamilton. Brian reminded the audience not to overlook Robotic Process Automation. This was the only reference to RPA that I heard, and it got me thinking of the broader AI context. A typical journey might look something like this:
- Data Wrangling
- Predictive Analytics
- Process Automation
- Data Engineering
- Machine Learning
- Model Ops
- Generative AI
Gen AI might appear in an automation role, as chatbot or search engine, but – certainly for financial services, which is swimming in metrics – the early use cases will be predictive.
While McKinsey estimates the impact of Gen AI around $3 trillion per year, this is on top of $11 trillion for non-gen AI and analytics. The McKinsey study is here, and the Microsoft maturity framework is here.
Conferences like this are a great way to see what other people are doing with AI but, in the end, you must decide how best to deploy it toward your own business needs. It’s a vehicle, not a destination.
