Category: F&I Products

AI-Based VSC Risk Rating

I have been working on a startup that will use artificial intelligence to rate vehicle service contracts.  For a VSC provider, increased accuracy means sharper pricing and, potentially, lower reserves.  Outsourcing this work to a specialist bureau means reduced costs, too.  Our business model is already used for risk rating consumer credit, and the technology is already used for risk rating auto insurance.

In this article, I present an example using auto insurance data.  If you would like to see how our approach works with VSC data, please get in touch.  We are currently seeking VSC providers for our pilot program.

The French MTPL dataset is often cited in the AI literature.  It gives the claims history for roughly 600,000 policies.  Of these, I used 90% for training and set aside 10% for testing.  So, the results shown here are not just “curve fitting,” but predictions against new data.

The Gini Coefficient

The challenge with insurance data is that most policies never have a claim.  This is known as the imbalanced data problem.  If you’re training an AI classifier, it can achieve 95% accuracy simply by predicting “no claim” every time.  You will want to use an objective function that heavily penalizes false negatives, and you may also want to oversample the “with claim” cases. 

The dashed line in the chart above represents cumulative actual claims, sorted in order of increasing severity.  This is called the Lorenz curve.  You can see that it’s zero all the way across and then, at the 95% mark, the claims kick in. 

The blue line is the Lorenz curve for the predictive model.  A good fit here would be a deep concavity that hugs the dotted line.  That would mean the model is estimating low where the actuals are low (zero) and then progressively steeper.

The Gini index is a measure of the Lorenz curve’s concavity.  This 0.30 is pretty good.  The team that won the Allstate Challenge did it with 0.20.  The downside to Gini is that it only tests the model’s ability to rank relative risks, not absolute ones.  I have seen models up above 0.40 that were still way off on actual dollars.

Mean Absolute Error

The key metric, to my way of thinking, is being able to predict the total claims liability.  This automatically gives you the mean, and Gini characterizes the distribution.  I like MAE because it represents actual dollars, and it’s not pulled astray by outliers (like mean squared error). 

Here, you see that the model overestimates by 1.2%.

You may be wondering why MAE is so high, when we are within $1.00 on the average claim.  That’s because all of the no-claim people were estimated at an average of $72.50, and they’re 95% of the test set.  The average estimate for the group that turned out to have claims (remember, this is out-of-sample data) was $130.70. 

Neural Networks

For claim severity, I trained a small neural net, including my own custom layers for scaling and encoding.  I really like TensorFlow for this, because it saves the trained encoders as part of the model.  You want to use a small neural net with a small dataset, because a bigger one can simply memorize the training data, and not be predictive at all.

This dataset has only nine features and, in fairness, a linear model would fit it just as well.  My code repo is now filled with neural nets, random forests, and two-stage combo models.  What this means for our startup is that we don’t have to hire a platoon of actuaries.  We can get by with a few data scientists using AI as a “force multiplier.”

Earlier this century, I played a key role in moving the industry to electronic origination.  At the time, it was clear that the API approach would liberate VSC pricing from the confines of printed rate cards and broad risk classes.  Each rate quote could be tailored to the individual vehicle.

As I said earlier, our approach is current, proved, and working elsewhere.  It’s just not being used in the VSC industry … yet.

Schrödinger’s Combo Product

NADA has recently published a model policy for properly selling F&I products, i.e., without running afoul of the Attorney General.  It includes the disclosure formerly known as the AutoNation Pledge, and a new procedure which seems to be taking the place of the old-school waiver form.  I say “seems” because there is no mention of the old form, which I believe has something to do with nuclear physicist Erwin Schrödinger.

Prior to the sale of a VPP, the Dealership will request the customer’s acknowledgement of the election to purchase or decline each selected VPP or VPP bundle.

As everyone knows, subatomic particles exist in an indeterminate state until they are pinned down by measurement.  For example, if you have a radioactive isotope of Cesium, you can’t tell whether it has decayed until you aim your Geiger counter at it.  Not only can you not tell what state the atom is in, it is not definitely in any state until you measure it.

To show how this contrasts with traditional physics, Schrödinger proposed the following thought experiment.  Imagine there is a cat in a box with the Cesium rigged to kill the cat when it decays.  According to the Uncertainty Principle, the cat is both alive and dead at the same time.

Similarly, the F&I waiver requires each product to be either accepted or declined.  You bought the dent protection, so it prints in the green column, but you turned down roadside assistance.  It prints in the red column.  To save a few dollars, you are willing to leave your family stranded.  Please sign here to confirm.

But what if dent and roadside – and key and windshield – are part of the same bundle?  You only bought one of the components, so it would be misleading to print it in the green column.  On the other hand, you are not going to confirm declining the bundle, because you did buy part of it.  So, in which column does this product belong?

Here are some ideas:

  • The menu system should account for the child products and print them individually on the waiver. It should also count them separately as product sales.
  • The menu system should print the coverage description, and the coverage description should state which components were accepted.
  • Providers should offer bundles all or nothing, and not allow them to be split up.

Unlike Schrödinger, you will not win the Nobel Prize for solving this one – but you can provide some guidance to your fellow F&I practitioners.  Click the link below to register your answer.

Penetration Chart with Bokeh

I have been honing my charting skills lately, because Bokeh is so amazing, and looking for practical applications (outside my stock trading hobby).  Here’s one I found recently.  This chart explores the timeless question, “are product sales off because the dealer isn’t supportive, or are vehicle sales off, too?”

I am thinking of protection products, but the same question could be asked of finance contracts or, indeed, anywhere you need to consider “penetration.”  That is, the percentage of vehicle sales that are also sales of your product.

Are product sales off because the dealer isn’t supportive, or are vehicle sales off, too?

In this chart, we consider year over year change in contracts relative to the change in vehicle sales for a collection of dealers.  Bubble size indicates the size of each dealership in sales volume.  We’ll get to bubble color in a minute.  Also, note the horizontal and vertical zero lines.

The dealers in the lower left quadrant have an excuse.  Riverside, for example, is down 30% in product sales.  When you call them, though, they’ll counter that they’re having a bad year.  Volume is also down, albeit only 11%.

The dealers in the lower right quadrant have no such excuse.  Downtown, for example, is also off 30% but on much improved vehicle sales.  So, we can infer that penetration has declined, and color them a darker shade of red.  Similarly, although contracts are up at National, they should be up more considering the good year they’re having.  So, orange.

O’Malley is green because, while contracts are off a bit, vehicle sales are worse.  O’Malley is doing the right thing and ramping up products to compensate for weak sales.  What the chart shows on the X and Y axes is straightforward enough, but it shrewdly assigns colors according to the change in penetration.

Bokeh is the visualization library Python programmers use instead of R or Matplotlib.  The color scheme here comes from running its red, yellow, green “linear color mapper” diagonally across the chart from lower right to upper left.  Dealers where penetration is unchanged from last year are yellow, like College and Bellevue.

Clampdown Looms for F&I Markup

NADA recently published a policy guide for protection products and we should commend the association for being proactive.  Highlights are below but, if you’re a practitioner, you must read and heed the full document.

  • Consistent presentation, i.e. use a menu
  • Prominent disclosures like the AutoNation pledge
  • Consistent, non-discriminatory pricing
  • Detailed waiver explaining any variance from standard pricing

Why?  Because otherwise dealers and lenders may be prosecuted.  NADA cites the $11 million Santander GAP settlement and the U.S. Bank deceptive marketing settlement.  I can see this going the way of dealer reserve.  Regulators will force lenders to restrict dealers’ discretion in setting markups.

NADA and others have warned on F&I markup since 2013, when the CFPB issued its first subpoenas on the topic, and last year the National Consumer Law Center published their report, subtitled: How dealer discretion drives excessive, arbitrary, and discriminatory pricing.

The chart above is one of several alleging discriminatory pricing in F&I.  As for lender pressure, the NCLC paints a big target on Ally Financial, reminding their readers that “state and federal authorities should investigate … and bring enforcement actions.”

Good operators will not have much to change for the model policy.  The recommended new waiver is a bit cumbersome, but the rest of it is already best practice, like menu selling.  The AutoNation pledge has been around for fifteen years.  Frictionless cancellation is discussed here.

In addition to regulatory pressure, there is also competitive pressure on F&I markup.  I’ll cover that in a later post.  On the bright side, AutoNation is near $2,000 a copy and their compliance has always been excellent.  So, no excuses.

Wanted: eCommerce Product Manager

Things are going well here at Safe-Guard, and I am looking to hire another eCommerce Product Manager.  Posting is here.  We need someone who can not only manage a shopping site but, as we are in the midst of a digital transformation, also establish the required support and fulfillment processes.

The eCommerce department manages the development and support of these properties, whether they are standalone web sites, dealer-site storefronts, or web services … 

The successful candidate will have solid product management experience, and maybe some digital marketing.  Agile development experience a plus.  Self-starter.  Relocation.   Salary commensurate with experience.

Analytics for Menu Presentation

Last week, I presented a single-column format for menu selling on an iPhone, with the glib recommendation to let analytics determine the sort order.  Today, I will expand on that.  Our task is to sort the list of products in descending order of their relevance to the current deal, which includes vehicle data, consumer preferences, and financing terms.

This sorting task is the same whether we are flipping through web pages or scrolling down the mobile display.  The framework I present here is generalized and abstract, making the task better suited to automation, but ignoring the specific F&I knowledge we all take for granted.  I’ll come back to that later.

For now, let’s assume we have six products to present, called “Product One,” and so on, and four questions that will drive the sorting.  Assume these are the usual questions, like, “how long do you plan on keeping the car?”

That answer will be in months or years, and the next one might be in miles, but we are going to place them all on a common scale from zero to one (I warned you this would be abstract).  Think of using a slider control for each input, where the labels can be anything but the range is always 0.0 to 1.0.

Next, assign four weights to each product, representing how relevant each question is for that product.  The weights do not have to be zero to one, but I recommend keeping them all around the same starting magnitude, say 1 to 5.  Weights can also be negative.

For example, if there’s a question about loan-to-value, that’s important for GAP.  High LTV will correlate positively with GAP sales.  If you word that question the other way, the correlation will still be strong, but negative.  So, now you have a decision matrix that looks something like this:

Yes, we are doing weighted factor analysis.  Let’s say that, for a given deal, the answers to our four questions are, in order:

[0.3, 0.7, 0.1, 1.0]

To rank the products for this deal we simply multiply the decision matrix by the deal vector.  I have previously confessed my weak vector math skills, but I am certain that Python has an elegant way to do this:

Product Two ranks first, because of its affinity for high-scoring Question Four.  Product Four takes second place, thanks to the customer’s response to Question Two – whatever that may be.  By now, you may have noticed that this is the setup for machine learning.

If you are blessed with “big data,” you can use it to train this system.  In a machine learning context, you may have hundreds of data points.  In addition to deal data and interview questions, you can use clickstream data, DMS data, contact history, driving patterns (?) and social media.

If not, you will have to use your F&I savvy to set the weights, and then adjust them every thousand deals by manually running the numbers.

For example, we ask “how long will you keep the car?” because we know when the OEM warranty expires.  Given make, model, and ten thousand training deals, an AI will dope out this relationship on its own.  We will do it by setting one year past the warranty as 0.1, two as 0.2, etc.  We can also set a variable indicating how complete the manufacturer’s coverage is.

Same story with GAP.  Give the machine a loan amount and a selling price, and it will “discover” the correlation with GAP sales.  If setting the weights manually, set one for LTV and then calculate the ratio for each deal.

Lease-end protection, obviously, we only want to present on a lease deal.  But we don’t want it to crowd out, say, wearables.  So, weight it appropriately on the other factors, but give it big negative weights for cash and finance deals.

I hope this gives some clarity to the analytics approach.  In a consumer context, there is no F&I manager to carefully craft a presentation, so some kind of automation is required.

All about Surcharges

Now here is an article for specialists only.  Menu system developers must know how to correctly acquire and present service contract rates, and surcharges are the most difficult feature.  Integrators and product providers also struggle with this, and I am writing today in hopes of establishing some industry norms.

We start with surcharge policy, from the provider’s perspective, and then data transfer and presentation issues for the menu system.

A surcharge represents an ad hoc increase to the claims risk, and therefore the price, of a service contract.  It lies outside the conventional pricing model, which is:

  • Risk Class – it costs more to service a Camry than a Corolla
  • Coverage – which parts and services are covered
  • Term – contract duration in months and miles
  • Deductible – claims risk is mitigated by a higher deductible

A surcharge is an extra feature tacked on to the pricing model.  For instance, the provider might want an extra $200 to cover a vehicle having a modified suspension, a turbocharger, or four-wheel drive, or if the customer intends to use the vehicle commercially.

Adding a flat dollar amount to the price is straightforward, but not especially accurate from a claims perspective.  That turbocharger will grow more risky as time goes on, so it is smart to have the surcharge amount increase with the term.

Note that you do not need to stipulate a four-wheel drive surcharge for Subaru.  They are all four-wheel drive, and so you can account for this risk in the vehicle classification.  Fixed (irremovable) features of the vehicle may be treated either as surcharges or risk classes.  In this example, four-wheel drive is handled as a class code bump.

Likewise, deductibles can be treated as surcharges.  This is an efficient way to represent them in a printed rate guide, where a choice of deductibles would mean many additional pages.  In the example below, the rate guide is printed with a base deductible of $50 and four more as surcharges.  Note that the surcharge amounts vary with the term mileage.

Warranty Solutions uses a similar approach, except that the surcharge amounts vary with the cost of the base contract.  They reckon that the risk associated with the vehicle, coverage, and term is already reflected in the cost, and so the surcharge should be higher on a higher-cost contract.  In my time as a consultant, I have seen everybody’s rate guide, and every possible way to handle surcharges.

It is important to recognize that a printed rate guide is just one way to represent the provider’s evaluation of risk.  As with Sapir’s theory of language, the provider’s actuary can only evaluate risks that can be expressed by the pricing model.

Where rates are returned via web service, there is no need to treat deductibles as surcharges.  They should be an explicit part of the pricing model, as above.  Where the VIN is supplied as input, likewise, there is no need to specify vehicle surcharges.

Many rate guides distinguish between “mandatory” and “optional” surcharges, but all surcharges are required to be levied where applicable.  Therefore, the usage I prefer is:

  • Mandatory Surcharge – We know it from the VIN, like a turbocharger
  • Optional Surcharge – We have to ask, as with commercial use

The user experience for a mandatory surcharge is simply to notify that we have already applied it to all rates in the web service response.  For optional surcharges, the menu system must provide a checkbox or some other way to apply it.

In either case, it is best for the web service to apply the surcharge to all rates in the response.  This allows for a smaller payload, and no chance for error.  The only reason to send rates both with and without an optional surcharge is if the menu system lacks the ability to request it up front.

Menu systems today already have user controls for the well-known surcharges, like commercial use, lift kit, snowplow, van conversion, warranty preload, synthetic oil, and rental coverage.  As a developer, I don’t like the idea of hardcoding these controls.  I would rather the menu system generate the controls at deal time, using a separate web service to obtain the list.

There is one kind of surcharge that must be included separately in the rate response.  These are additions to coverage which the F&I Manager may upsell at deal time.  The mockup below shows the addition of optional electrical to one grade of coverage, which is included with the higher grade.

Because the F&I Manager may toggle this surcharge dynamically, there is no alternative but to include it in the rate response.  This means an extra branch at the coverage node, assuming a tree structure, or else sprinkle the surcharge among the leaf nodes and make the menu system do the math.

  • Upsell Surcharge – We may choose it dynamically at deal time

Either way, dynamic surcharges will bloat the rate response.  The workaround we used at MenuVantage was to treat them as optional surcharges, above, and ask the F&I Manager to choose prior to rating.  I frankly hate dynamic surcharges, a prejudice from my menu days, but people evidently find them useful.

That about does it for surcharges.  If anyone has anything to add, in the spirit of setting industry norms, please write in.

Wanted: eCommerce Product Manager

Gartner Group says “the API is the product.”  I am looking for an experienced product manager who knows what Gartner Group is and why they say that.  The API in question is Safe-Guard’s collection of dealer-facing web services.  This is a topic I have worked on and written about extensively, as here, and now I plan to try the product manager approach.

The successful candidate will have solid product management experience, preferably with an API, and maybe some pragmatic marketing or agile development.  Software development experience a plus.  Self-starter.  Relocation.   Salary commensurate with experience.

Wanted: Experienced F&I Trainer

I am in the process of creating an eCommerce department for Safe-Guard.  Regular readers know that I specialize in creating new organizations, and my record is pretty good.  The training function, which is also a kind of sales function, is likely to grow.  So, this is an opportunity to get in early.

The job is to train all of the F&I managers who sell products administered by Safe-Guard, and ensure they know how to present them properly using any of the top ten menu systems.  For one person, at least to begin with, this will be a challenge.  We are in thousands of dealerships.

Thus, the successful candidate must have the skill and temperament to leverage the resources of our affiliated agents, vendors, manufacturers, and dealer groups.  Self-starter.  Travel.  Proficiency in F&I procedures and software, notably menu systems.  Salary commensurate with experience.

Stop Using Combo Products

I have had a hand in designing a few menu systems over the years, and I have always disliked combo products.  You know what I mean: the VSC form, plus maintenance and PDR, on which Marketing has found an extra square inch to offer road hazard.

Menu people hate combo products because the whole point of menu selling is for the F&I Manager to combine products into packages, not the combinations defined by the provider’s form.  What if she wants to sell the factory’s VSC, but her own choice of ancillary products?

One cavil I sometimes hear is the definition of “a product,” but this is straightforward.  If it can be sold separately, like key protection, then it’s a product.  If it always rides on another contract, like car rental, then it’s not.

What I try to tell my menu clients (and reinforce with my API clients) is this:

  • The unit of work for presentation is the product
  • The unit of work for contracting is the form

The correct data structure thus has discrete products at the top level, then coverages with their rates, and form codes at the bottom.  Obviously, you can have different forms based on coverage, and you can have the same form for multiple products.  Then, in the contracting phase, you collect the products onto the forms as indicated.

Combo Products

Combo products persist because providers legitimately want to reduce the number of forms they manage.  The two-phase approach solves this.  Also, there are old-timers who design products based on the form.  I have even seen F&I shops where the completed contract form is used as a selling tool.

The package discount is the only serious challenge to the menu system.  A workaround here is to include a phantom product with no display and a negative price – although that may be as much work as developing an explicit feature.  Of course, if the manager chooses to discount a package other than one subsidized by a provider, then that discount is her responsibility.

I’ll close with an exception to the rule or, rather, a refinement.  Menu systems are compromised when we mistake forms for products.  On the other hand, there is a practical limit (six) to the number of products offered on a menu.  So, I can see the logic in a product that combines dent, coatings, windshield, and road hazard – especially PDR and windshield, if you think about how the services are delivered.

In this case, we are not merely combining products based on a form.  These products hang together in the same semantic class, appearance protection, and may indeed use separate forms.