Client
A national agency responsible for approving vehicle test stations and supervising the periodic roadworthiness test
Sector
Vehicle Testing & Road Safety
Engagement
Peer-group statistical monitoring of test station results to prioritise inspector visits - multi-quarter programme.
What the client needed
Tens of thousands of tests a day flow through thousands of approved stations, and a small number of those stations were passing vehicles that should have failed - sometimes for money, sometimes out of habit. Supervision relied on scheduled visits, mystery-shopper vehicles and complaints. All of them are slow, and the complaint route only ever catches problems a motorist happens to notice. Inspectors also had no fair way of comparing a rural garage that tests older vans with an urban one that tests hire cars, so crude league tables kept flagging the wrong places.
How we worked
- Loaded test-level records - outcome, defect categories, duration, vehicle age and class, retest gaps - into a monitoring store refreshed nightly.
- Built peer groups by vehicle mix and local market rather than by region alone, so a station is only compared with stations testing a genuinely similar workload.
- Scored each station on several independent measures, including pass rate, share of advisories, test times and repeat vehicles, and required more than one to move before raising a flag.
- Gave inspectors a ranked worklist with the reasons written out in plain language, plus the tests behind each score, so a visit could be planned from the screen.
- Kept the output advisory. Nothing suspends a station automatically; the decision stays with an inspector and the agency's formal process.
- Recorded visit outcomes back into the system so measures that produced too many dead-end visits were re-weighted.
Measured results
All figures verified with the client. Station names and enforcement detail withheld in line with our standard confidentiality terms.
- Inspector visits are now driven mostly by the ranked worklist, and a clear majority of the visits it produced in the first year found something worth formal action or advice.
- The time between a station's behaviour changing and a visit being scheduled fell from months to a few weeks.
- Comparing like with like removed most of the false alarms that the old crude comparisons produced for stations serving unusual local markets.
- Inspectors report spending less of their week driving to stations that turned out to be fine.
- The agency is evaluating the same approach for supervising other approved-provider networks.
"We always knew which stations felt wrong to the people who'd been in the job twenty years. What we couldn't do was show a newer inspector why, or spot the ones that started drifting last spring. Now the screen says it, and the reasons hold up when we knock on the door."
Working on something similar?
If this engagement looks like the kind of problem you are facing, we would be glad to compare notes by email.
Context and constraints
A test station is a private business that holds a public licence. That makes supervision delicate: an accusation based on a number that turns out to be innocent damages a legitimate garage, and the agency has to be able to defend every visit it makes. So the platform was never allowed to produce a verdict, only a reason to look. Every flag had to be explainable to an inspector, and to the station owner if it came to that.
Why the peer group was the hard part
Raw pass rates are a poor signal. A garage next to a used-car auction sees far more borderline vehicles than one serving a retirement village. Most of the build effort went into constructing peer groups that made comparisons fair, and into testing them with inspectors who could say immediately when two stations were not really alike. We ran the first versions in shadow beside the existing schedule for several months before anyone changed a diary.
Requiring more than one signal
Any single measure will misfire on some honest station eventually. Requiring several independent measures to move together cut the noise sharply and matched how experienced inspectors already thought - short tests and low failure rates and the same few vehicles coming back, for instance. It also made the written explanation on the worklist almost write itself.
Lessons learned
The first lesson was that the credibility of a supervision tool is decided by the fairness of its comparisons, not the sophistication of its statistics.
The second was that shadow running was worth the delay. It let inspectors argue with the system before their reputations depended on it.
The third was that feeding visit outcomes back in was not a nice extra. Without it the weights would have gone stale within a year.
If you supervise a network of approved providers and rely on complaints to find the bad ones, we would be glad to talk. Email sales@halfteck.com.