Software Development at RumbleOn

I have some jobs open right now and I am also busily documenting our Software Development process for the auditors, so I thought this might make a good blog post.

The magic starts in our Product Management department.  We treat all our software as program products with roadmaps, features, and release plans.  This includes our internal operational systems as well as omnichannel retail.  I have long been a proponent of this approach, even for APIs.

Our vision is to have an integrated pipeline from the survey portal all the way to deployment.

Artifacts like requirements, concurrence, priorities, and customer feedback are maintained in Aha! Roadmaps.  Product Management is also design driven, but I digress.  This post is supposed to be about Software Development.

Software Development includes four agile teams.  Most of the staff are here in Dallas, with some work offshore in India and Eastern Europe.  See my post on sprint planning with time separation.  The teams manage their work with Jira.

Jira synchs in both directions with Aha! Roadmaps, and will soon synch with Circle CI for deployment.  Our vision is to have a seamless pipeline from Aha’s survey portal, through feature planning, all the way to deployment.  For omnichannel retail, we add feature flagging.  We are not quite to the point of doing statistical testing and audience segmentation, but we’ll get there.

Agile teams manage their own deployments and feature branches, but releases are coordinated with Product Management and moves to production are regulated by the Architecture team.  This is designed to be a choke point for secure change management.  The Architecture team also maintains a reference library of artifacts for standardization, like federated identity.

If you’re interested in managing one of our agile teams, here is the link.  Work is onsite in our Dallas location, corner of MacArthur and 114, convenient to fine dining and Sunstone Yoga.  The facility is urban industrial, with terrazzo floors and exposed béton brut.  No remote work, sorry.  That’s so 2020.  As a manager, we’ll need you onsite.

We are looking for local talent, preferably Texans of the DFW variety.  This is to avoid relocation challenges.  There is also a certain corporate culture that comes from selling 80,000 street bikes, dirt bikes, and ATVs each year.

RumbleOn does not discriminate on the basis of race, sex, color, religion, age, national origin, marital status, disability, veteran status, genetic information, sexual orientation, gender identity or any other reason prohibited by law in provision of employment opportunities and benefits.

Sprint Planning with Time Separation

Poland TimeHere is a short post on a practical problem for my readers who use agile.  My client is in Dallas, and most of the developers are in Poland.  That’s a seven-hour time difference.  We have enough overlap that daily scrums are not a problem.

“What I’m fixing to do today,” in Texas, comes at the end of the team’s workday in Poland.  This is a little weird, but not a problem.  Sprint planning is a problem.

With everyone in the same time zone, I like the rhythm of closing the sprint on Friday, doing the review, and then sprint planning on Monday.  If your idea of “sustainable pace” includes working weekends, the team gets that weekend off.  Then, we start Monday with a fresh scrum board.  This is probably everyone’s favorite rhythm, if you have the luxury of cotemporal teams.

By the way, I write “cotemporal” here instead of “collocated” because it’s the time zone that matters, not physical distance.  I know of at least one F&I company based in New York with developers in South Florida (an F&I “tech cluster” thanks to AutoNation and JM&A).

Replacing the initial eight-hour session with two separate four-hour sessions conducted over consecutive days is more practical.

With widely separated time zones, the problem is that you can’t leave the scrum board empty.  Unless the next sprint is planned immediately, one team or the other will have a day with no work.  So, we have adopted what Mike Cohn calls the two-call method.  The quote above is from his book, Succeeding with Agile.

We close the sprint on Friday, do the review, and begin sprint planning with the Dallas cohort.  We have the scrum master here, the product owner, and the lead developer.  So, that’s a quorum.  The downside is that the Polish team misses the initial discussion of the sprint goal.  Cohn lists pros and cons in the book.

Relating the big picture falls to the lead developer, who conducts a second session on Monday during the overlap period, and develops the sprint plan.  This approach doesn’t allow much debate over scope, so it places a premium on good stories with good estimates.  For more on agile, read my post Sales Driven Development (or buy Cohn’s book).

Sales-Driven Development

Larman BookI invented agile development. No, that’s not as cocky as it sounds. Every development manager I know has been trying to reduce cycle times, forever. At BMW, we did quarterly releases of call center software, and that was on client-server technology. As a profession, we have always been as agile as possible given the tools of the day.

What we call “agile development” is the ability to release software early and refactor it later. People generally overlook the high cost of refactoring, but I’ll save the critique for a later post. You can read The End of Agile, over at Effective Software Design. What I would like to present here is a practical approach that served us well at MenuVantage, and which is still relevant today.

We were operating in a highly competitive, fast moving market. Our sales people would come in and say they had just lost a sale because Brand X had some feature we didn’t. Better yet, they would have closed the sale on the promise that we would have it in next release. The developers complained about “sales driven development,” which, today, sounds like it could be a book title.

Our cycle ran from development on the local laptops, through automated daily builds, to the hosted production servers – every six weeks. I know that the scrum literature talks about producing “shippable” code every two weeks, but few shops actually move to production that frequently unless they are in a consumer market, running AB tests. I assume that most of my readers are developing business software for dealers and finance sources.

One innovation in our process was a second QA environment, on the company extranet, where product owners could preview the work in progress. Business requirements were thin, as I’ll explain later, and the extra environment gave us some ability to iterate within the six week cycle. I dubbed it “rolling QA,” in the process document.

When we deemed that it had rolled enough, we froze this code and moved it to production.   Meanwhile, “nightly QA” would already be in use for the next release (see diagram). We also maintained a second copy of production, called “post-prod,” that we could use just in case we should ever need to diagnose and fix a critical problem in production.PipelineAs each release went live, the product owners would meet to negotiate the content of the next release. This expression, “product owner,” is another thing that bugs me about scrum. In this case, though, the meeting included the legal owners of the company. We might differ over development priorities, but we had absolutely congruent goals for the company.

In preparation for the meeting, we would have pulled together a short list of the highest priority change requests, and estimated them – in hours. We did not use “complexity points,” because sales and finance people are much happier thinking in hours. For one thing, it facilitates a discussion about how many developers there are and how many hours they work. If you are uncomfortable having this discussion, trust me – using an arcane metric instead of hours will not make it easier.

Within the development team, we did use complexity to derive each estimate, by counting how many modules, pages, queries, etc., had to be changed. Contract developers have bid projects this way forever. I am thinking, in particular, of a spreadsheet used by Ernst & Young’s Advanced Development Center. Over time, the conversion factors are updated, so “institutional learning” about the estimates is captured.

We used conventional bug tracking software to manage the backlog. They get better every year. I will note that the last scrum system I used, JIRA, is a repurposed bug tracker. This is part of the reason why I find agile is better for enhancing an existing product than it is for the initial release – but that’s the other post.

My job was to collect high level requirements, write specs, and groom the backlog. Requirements were at the level of, “we need rollbacks so that we can do Sign & Drive.” This is actually not a bad formulation for a scrum story card. Readers will recognize the complexity concealed within this simple request – the UI changes and the math, lots of math.

The goal of the planning meeting was to distill the “short list” into an even shorter list of requests we could accommodate in the release. Don’t be surprised if your short list becomes a LIFO stack. That’s human nature. If something was hot two cycles ago, and it was never scheduled, then it probably isn’t hot anymore.

Development would start the meeting by presenting the short list and announcing how many hours we had available. Initially, we might have been paring a 3,000 hour short list down to 1,500 for the release. After a few cycles, though, everyone was able to predict what the final slate would be. The meetings became shorter and less contentious.

This is where we really enjoyed the advertised benefit of agile – and what motivates me to endorse this approach. The group learned that if we didn’t get all our requests into this release, we could rely on the next one, and the sales team developed a feel for how much they could promise.