Know Your Time Series

I recently saw this chart (below) from PNAS.  It’s one of those popular psych studies that asks “does having more money make people happier?”  They discovered that higher annual income does indeed make people happier, but in a logarithmic relationship.  This reinforces something I wrote in Sensitivity Testing Model Assumptions, namely: know your time series.

In today’s post, I’ll explain about log scales using an example from stock trading, and then circle back to the happiness data.  I’ll wind up with an example from my own experience, contrasting exponential versus polynomial functions.  You may also want to check out my earlier posts on seasonal adjustment, and Bayesian probability.

“When interpreting these results, it bears repeating that well-being rose approximately linearly with log(income), not raw income”

Reported “life satisfaction” and “well-being” increase linearly with log income, not straight income.  That is, the next notch up in happiness requires an order of magnitude more money.  Previous studies had found a plateau around $75,000, with little or no increase in happiness after that.

So, this is a new finding – or is it?  Take another look at that income scale.  If that were a straight scale, the chart would show diminishing returns from additional income.  To enjoy steadily increasing happiness, you have to earn exponentially increasing income.

Log Scaling for Stock Charts

Here is a chart of Carvana during 2018, when the stock was rising rapidly.  Comparing the tiny candles in March with the longer ones in June and beyond, you might conclude that the stock had become more volatile.  The average daily trading range increased from one dollar to three over the period.

But a dollar when the stock is at $20 is not the same as a dollar when the stock is at $60.  To have the candles represent percentage change, you must set the price scale to logarithmic.  See, in the chart below, how the price intervals get closer together as they proceed up the scale.

Whenever a stock chart covers a wide price range, you’re better off using a logarithmic scale.  You may recall from school that adding logs is the same as multiplying the numbers.  So, a linear scale shows additive change, and a log scale shows relative change.

log ab = log a + log b

Take another look at the first Carvana chart above.  Stock traders call that “going parabolic.”  Parabolic growth, also known as “quadratic,” is another rapid growth trend, easily confused with exponential growth.  I did a quick regression analysis, and both models fit the Carvana data pretty well.

Pro tip: Never use “exponential” to describe something that’s not a time series.  Some people seem to think it just means “big,” as in “last month was exponential!”

The point to “know your time series” is to understand the mechanisms underlying your data.  Exponential growth comes from compounding, like if you increase sales by ten percent, and then you increase the new, higher, base by another ten percent – and you keep doing that.

Steadily Increasing Happiness

I’ll provide an example of quadratic growth later, but first let’s finish up the PNAS chart.  I think of this as a time series because I picture someone earning steadily more income over their career (the data is actually different people at different income levels).

When I say “steadily more income,” I mean exponentially.  Note that each tick mark on the PNAS income scale doubles the value.  This is a log scale, like the Carvana price scale, above.

Many real-world metrics are based on log scales, like decibels and the Richter scale.  An earthquake of magnitude 6.0 on the Richter scale is ten times as powerful as a 5.0.

The chart below shows what this blessed career looks like.  If you start making $15,000 at age 18 and double your salary every six years or so, then you will experience steadily increasing “well-being.”  My red line is the same red line as in the PNAS chart.

I think showing the data as someone’s career is a good way to tell the story.  Income and well-being are shown together, with straight scales, and mediated by the hypothetical age.  On the other hand, the correlation has disappeared.  To show that, we must apply a log scale to the income series:

This is why the authors make clear that, to enjoy steadily increasing (linear) happiness, you must earn steadily increasing (exponential) income.  To put it another way, if you only earn increasing (linear) income, then you will have only increasing (log) happiness.

Many real-world metrics are based on log scales, like decibels and the Richter scale.  An earthquake of magnitude 6.0 on the Richter scale is ten times as powerful as a 5.0.

Polynomial versus Exponential Functions

Once upon a time, way back when databases had size constraints, I observed that parabolic growth in BMW Financial lease transactions would pose a danger to the database.  I ran a regression analysis, calculated when the database would fail, and sent a memo to my boss.

I also worked out a mitigation strategy, but let’s stick with, “the database will blow up on April 21,” for dramatic effect.  Instantly my office filled up with expensive auditors and consultants.

“No, it’s not a malfunction.”

“No, it’s not growing exponentially.”

Lease transactions were growing quadratically, which is why I chose this example.  If new leases are steady at 1,200 per month, that’s a flat line.  Total rows in the lease table will thus increase by 1,200 per month.  That’s a sloping line.

Now, if each lease generates roughly two transactions per month, then total transactions will be a parabola.  Readers with a little calculus will recognize this as integrating, twice, from the constant rate of new leases to the second-order rate of transactions.

This is the essence of “know your time series.”  The regression analysis showed quadratic growth, unequivocally, and it was also supported by how we expected the data to behave.

The auditors milled around for a while, charged us about a million bucks, and decided I was panicking over nothing because the database wasn’t really going to blow up until April 29.  Not to mention my mitigation strategy.

Big O Notation

In the example above, I showed that a linear function of a linear function is a parabola, also known as a “quadratic” or second-order polynomial.  Compose another linear function on top of that, and it’s a third-order polynomial.

Excel has a handy feature, shown below, for fitting polynomial functions of different order.  This is a little dangerous, because you can easily find a fit without thinking about it.  It’s not enough to get a good R-Square (fit) value.  You must understand why the data behaves as it does.

This “order” thing is sufficiently important to data analysts that they have a notation for it, called “Big O.”  The quadratic example we just worked through would be O(n2) or “order n-squared.”  I notice that Excel will also fit a Power Law, or “Pareto” series.  That will have to wait for a later post.

Software Development at RumbleOn

I have some jobs open right now and I am also busily documenting our Software Development process for the auditors, so I thought this might make a good blog post.

The magic starts in our Product Management department.  We treat all our software as program products with roadmaps, features, and release plans.  This includes our internal operational systems as well as omnichannel retail.  I have long been a proponent of this approach, even for APIs.

Our vision is to have an integrated pipeline from the survey portal all the way to deployment.

Artifacts like requirements, concurrence, priorities, and customer feedback are maintained in Aha! Roadmaps.  Product Management is also design driven, but I digress.  This post is supposed to be about Software Development.

Software Development includes four agile teams.  Most of the staff are here in Dallas, with some work offshore in India and Eastern Europe.  See my post on sprint planning with time separation.  The teams manage their work with Jira.

Jira synchs in both directions with Aha! Roadmaps, and will soon synch with Circle CI for deployment.  Our vision is to have a seamless pipeline from Aha’s survey portal, through feature planning, all the way to deployment.  For omnichannel retail, we add feature flagging.  We are not quite to the point of doing statistical testing and audience segmentation, but we’ll get there.

Agile teams manage their own deployments and feature branches, but releases are coordinated with Product Management and moves to production are regulated by the Architecture team.  This is designed to be a choke point for secure change management.  The Architecture team also maintains a reference library of artifacts for standardization, like federated identity.

If you’re interested in managing one of our agile teams, here is the link.  Work is onsite in our Dallas location, corner of MacArthur and 114, convenient to fine dining and Sunstone Yoga.  The facility is urban industrial, with terrazzo floors and exposed béton brut.  No remote work, sorry.  That’s so 2020.  As a manager, we’ll need you onsite.

We are looking for local talent, preferably Texans of the DFW variety.  This is to avoid relocation challenges.  There is also a certain corporate culture that comes from selling 80,000 street bikes, dirt bikes, and ATVs each year.

RumbleOn does not discriminate on the basis of race, sex, color, religion, age, national origin, marital status, disability, veteran status, genetic information, sexual orientation, gender identity or any other reason prohibited by law in provision of employment opportunities and benefits.

Reverse Omnichannel

If digital retail is so great, then why does Apple have stores?  Shipping a six-ounce phone is nothing compared to delivering a car, and yet these iconic stores.  This, of course, is the true meaning of “omnichannel” retail – meeting your customer wherever they choose. 

Digital native retailers, in fact, are advised to add physical stores, even if they’re selling only sneakers or eyeglasses.  McKinsey advocates this as a way to gain cheaper traffic – ironic, considering the popular misconception that digital retail has lower costs. 

Brick and mortar car dealers have service departments which can keep them afloat in a recession.  They also have an easier time selling F&I products.  So, no surprise that online car dealer Driveway has lately opened a showroom, except … Driveway is the online brand of Lithia, a public dealer group with over 250 stores. 

If a Driveway customer in Oregon wants a vehicle that’s in stock in Texas, it’s shipped to the Portland store and delivered to the customer.

That’s right, the online brand of the leading dealer group now has its own store – with no cars.  This makes perfect sense to me.  I bought my last car online, in the dealership.  The salesperson ran the same configurator I would run at home, adding value with her knowledge of the product.

Omnichannel Auto Retail

Here is a brief history of how we got here, with links to contemporaneous coverage on the blog.  Schematically, the omnichannel evolution looked like this:

I started writing about digital retail way back in 2015, with a two-part post on Design Concepts for Online Car Buying.  I didn’t manage to land a gig building one, but I got the next best thing.  My job running e-commerce for Safe-Guard put me in touch with emerging leaders like Roadster and Accelerate.

These systems allow dealers to retrofit digital retail into their existing websites, while public groups Lithia, Asbury, and CarMax developed their own.  I covered the market for digital retail software from a few different angles.  See here, here, and here

Dealers invested in digital retail, but they didn’t always get the desired results.  Software vendors were the first to spot the disconnect between process and technology.  Roadster started writing about omnichannel in 2018.  Cox’s Mike Burgiss exhorted dealers to “sell the car, not the appointment.”  

The true revenue performance of a retailer’s online channel can be understated by up to 100 percent, or even more if not accounting for the influence online has on offline.

Around this time, I was writing about a Best Buy model for auto retail.  McKinsey linked the two concepts in this 2021 article.  For me, their most important observation is that our metrics don’t always give proper credit to the online channel. 

Digital Native Car Dealers

Digital natives Vroom and Carvana missed the memo about having a physical presence.  Hell, even Amazon has retail stores.  Lithia and CarMax are more like “digital immigrants.”  Driveway going back to its roots and opening a physical store reminds me of “reverse ETL” from Data Engineering.

Data Engineers spend a lot of effort extracting, transforming, and loading (ETL) data for use in analytics.  Then, we often find it useful to take the cleaned-up data and push it back into the transactional system whence it came. 

Another analogy might be how elephants evolved from seagoing mammals and back to land again, or how computing power was centralized in the mainframe era, and now recentralized in cloud services … but “reverse omnichannel” makes a better title. 

Cost Accounting with Scrum

Here is another in my occasional series on the finer points of scrum.  See also Sprint Planning with Time Separation.  Cost accounting seems inimical to scrum, philosophically, and also infeasible.  We use story points for a reason, and then we let the team discover its velocity through experience.  Neither of these numbers is readily convertible into dollars, but that’s exactly what we’re going to do.

If the latest sprint delivered 70 story points, those points are worth $402 each.

Our goal is to calculate how much was spent to develop a certain feature.  This can be to support a cost-benefit analysis, to track development as a capital investment, or to claim an R&D tax credit.

The team’s velocity is the number of story points it can complete in one sprint, typically two weeks.  Velocity changes from one sprint to the next, but the key is – we know the velocity of the most-recent sprint, and that’s the one we need to account for.

We also know how much it costs to run a sprint.  Let’s say that we have a seven-person scrum team with an aggregate base salary of $677,500.  Adding an 8% burden rate and dividing by 26, we calculate that the cost per sprint is $28,142.

So, if the latest sprint delivered 70 story points, those points are worth $402 each.  Now, let’s say that two capital projects absorbed 65 of the 70 points, plus a stray five-point story that fixed a bug or something.  It was a regular expense.  Here is the cost allocation:

It’s easy enough for the scrum master to load these figures into an accounting system at the end of each sprint, but it does require each user story to be tagged with the project it represents.  If you’re using Jira, it’s best to group the stories into epics, which represent new features, and include the capital project identifier (i.e., an account number) on the epic.