Deep learning changed what’s possible in computer vision. Convolutional Neural Networks pull real patterns out of complex image data, from medical diagnostics to identifying leaf disease in the field.
None of that changes an uncomfortable fact the industry mostly avoids: the same architecture, trained twice on identical data, gives you two different accuracy numbers. Weight initialization, data shuffling, and dropout all inject randomness into every training run.
The standard response isn’t to fix that. It’s to run a dozen experiments and keep whichever one scored highest.
That’s not engineering. It’s betting on a lucky seed.
Peak accuracy isn’t process stability
Non-deterministic training is often the real culprit when high-stakes models fail in the real world.
Mean accuracy hides the variance sitting underneath it. A model can post a strong benchmark number while several of its training runs perform far worse, and the average never shows you that.
A model that tops the leaderboard might just be the outlier run. Deploy it in a clinic or a field, and the accuracy that looked solid on a spreadsheet may not hold.
Most ML frameworks treat each training run as its own isolated event rather than a process. Without statistical control on that process, there’s no way to tell whether an accuracy gain came from an actual architectural improvement or from the random seed landing well.
A manufacturing plant producing usable parts only 10% of the time would be shut down. Machine learning routinely ships models built the same way, on uncertified, high-variance pipelines, and calls it standard practice.
Statistical Process Control isn’t new; it’s borrowed.
Quality engineering solved variance management with Statistical Process Control decades before deep learning existed. Applying it to training pipelines isn’t a stretch. It’s overdue.
Instead of hoping for a good run, the training pipeline gets governed the way a manufacturing line is governed.
“`
[ Baseline Measurement ] ──► [ Factorial DOE ] ──► [ Cpk Certification ]
(i-mR Control Charts) (Lock Hyperparameters) (Deployment Gate)
“`
Track jitter with i-mR control charts
Individual-Moving Range (i-mR) charts map accuracy across multiple training runs and set statistically determined Upper and Lower Control Limits (UCL/LCL). That separates background noise, the common-cause variation every run has, from a real defect or a real improvement, the special-cause variation worth acting on.
Lock hyperparameters with Design of Experiments
A 2³ Factorial Design of Experiments tests how the settings that matter actually interact, instead of guessing at combinations one at a time. Running Dropout Rate, Weight Decay, and Data Augmentation against each other shows which combination minimizes run-to-run variance and locks the pipeline into a state that resists random seed noise.
Certify models with Process Capability (Cpk)
Peak accuracy alone shouldn’t decide whether a model ships. The Process Capability Index (Cpk) checks whether the training process behind the model is stable and reproducible, not just whether one run happened to score well. A slightly less accurate model with near-zero variance beats a volatile, high-accuracy outlier every time it actually has to run in production.
Certified, not lucky
Deep learning can’t scale into mission-critical work on luck. Treat model training as an industrial process governed by SPC charts, DOE, and Cpk, and the non-deterministic guessing game stops being the default.
The goal isn’t a model that’s accurate by chance. It’s one that’s reliable by design.
Reference & Citation Notice
This blog post is adapted from doctoral research in Computer Science:
Akin-Tepede, Oladipupo. “Statistical Control of Stochastic Training-Run Variation.” PhD Dissertation, Department of Computer Science.
