Collision parts are replaced after an accident, so they sell rarely and without much of a pattern. That makes them the hardest thing in a spare-parts catalogue to forecast, and the easiest to get wrong.
For the final assignment of the Career Accelerator at the University of Cambridge we were given an employer project. The client, a global automotive distributor, tasked us with improving how they forecast demand for collision parts: the body and exterior components replaced after an accident, which by their nature sell rarely and unpredictably.
Working as part of a five person team, we went about tackling the problem. I owned two workstreams, the external data and the lumpy demand. The engagement is under a non-disclosure agreement, so what follows is the method, not the client's results. Every figure on this page comes from public datasets.
The team needed external variables in a state they could actually model with, and needed to agree on one version of them. The dashed step is the one I would argue for hardest.
The split rests on two questions: how often does a part sell, and how steady is the quantity when it does. That gives four classes, and they do not respond to the same treatment.
Real demand, not illustrations: three years of monthly sales for one representative part in each class, from the open car-parts dataset, on a common vertical scale. Lumpy and intermittent both sell rarely; the difference is that lumpy jumps around in size when it does, while intermittent is roughly the same quantity each time.
Lumpy was the hardest class in the catalogue and the one I took on. These three parts all sit in the same class, and all three would be handed the same model.
Three real parts from the same demand class, three years each. Same classification, three completely different situations.
No history to learn from, and about to matter. A model fitted on its past has almost nothing to work with.
The steady case the models are actually built for, and the only one of the three they handle well.
Stopped selling eighteen months ago, and is quietly being forecast, and stocked, as though it might come back.
The method was built for one client's lumpy parts. The question I wanted to answer next was whether it worked elsewhere, so I extended it from lumpy to all four demand classes, widened the candidate field so each class had models suited to it, and ran it end to end on public data where the results could be published and checked.
| Segment | Products | WMAPE | vs naive | Bias |
|---|---|---|---|---|
| Intermittent | 13,453 | 28.55 | 0.998 | −14.2% |
| Smooth | 12,765 | 25.63 | 0.981 | −9.7% |
| Lumpy | 1,701 | 38.45 | 1.018 | −23.6% |
| Erratic | 971 | 40.44 | 0.912 | −18.0% |
The headline accuracy is healthy. A WMAPE in the twenties on retail demand is a respectable number and it would be easy to stop there.
What "vs naive" measures. The naive forecast is not zero. It is the simplest thing a planner could do without any model at all: carry the recent level forward. That column divides my engine's error by the naive forecast's error, so 1.018 means my engine was 1.8% worse than just carrying the level forward. The column is not asking whether the forecast is good. It is asking whether the machinery was worth building.
On that measure M5 is humbling. Three of the four segments land between 0.98 and 1.02, and lumpy is slightly worse than the simple approach. M5 is not a hard forecasting problem: the typical product sells around ten units a month in most months, so carrying the level forward already works well. Good numbers, earned mostly by the data.
So I took it somewhere sparse on purpose. The open car-parts dataset is the same kind of catalogue the client engagement dealt with, and nothing like M5, even though both label a segment "intermittent".
| Segment | Parts | WMAPE | vs naive | Bias |
|---|---|---|---|---|
| Intermittent | 1,515 | 90.19 | 0.891 | −27.1% |
| Lumpy | 242 | 86.33 | 0.812 | −33.0% |
| Smooth | 15 | 83.97 | 0.851 | +7.8% |
| Erratic | 4 | 70.52 | 0.860 | −29.2% |
Greyed rows are too small a population to conclude anything from. Bias is signed: negative means the forecast sits below what actually sold.
Those percentages look like a broken model, and this is the part worth slowing down for. A typical car part sells one unit, roughly twice a year. Percentage error divides by demand, so at a third of a unit a month, being off by a single unit is a 300% error. The floor is set by the data, not by the model.
An oracle, handed each part's true average for the test window it had not seen, scores 119.8% WMAPE on the lumpy segment. My engine, with no such advantage, scores 86.33%.
A forecast with perfect hindsight of the answer does worse, on this metric, than the one built without it. That is not a claim about the model being clever. It is proof that percentage accuracy on demand of one or two units a month is measuring the catalogue, not the forecast, and that ranking this kind of data by WMAPE will point you at exactly the wrong problems.
Pooled WMAPE is a single number for a whole segment, weighted by volume, so a handful of high-turnover parts can set it for everyone else. A planner does not manage a segment. They manage lines.
Per-part error is a three-month rolling WMAPE for each part; the pooled figures in the tables above span the full evaluation horizon and are volume-weighted, so the two are related but not the same calculation. Car-parts smooth and erratic are omitted at 15 and 4 parts.
On car-parts intermittent the pooled figure is 90.19. The median part comes in at 38.5, seven in ten are under 50%, and one part in a thousand is at or above 100%. The segment-level number describes almost none of the parts in it.
The comparison with M5 is the part I did not expect. On the per-part view car parts is the more dependable of the two: essentially no part exceeds 100% error, while on M5 between 9% and 20% of parts do, depending on segment. The dataset with the far worse headline number has the better distribution underneath it, and only the per-part view shows that.
Error relative to a naive forecast. Left of the line the engine is earning its place; right of it, the simple approach was better. Segments too small to judge are left out.
Car parts, the dataset with the ugly percentages, holds the strongest result on this measure. M5, the dataset with the pretty ones, barely registers.
Car-parts lumpy at 0.812 is the best result across either dataset, and it comes from the segment with the worst headline number. M5 lumpy, at 1.018, has a WMAPE less than half as large and is beaten by carrying the level forward. Judged on accuracy alone the ranking inverts, and you would put your effort into the catalogue that never needed it.
The conclusion is a planning one rather than a modelling one. A forecasting engine should be pointed at the part of a catalogue where it demonstrably beats the simple alternative. Everything else belongs on a min/max reorder policy with safety stock, where the question is how much cover to hold rather than what next month looks like. Deciding which lines fall on which side is worth more than another point of accuracy on lines that were never the problem.
The method was developed on a client engagement covered by a non-disclosure agreement. Every figure on this page is reproduced on public datasets. No client data, results or identity appear here.