Fraunhofer FKIE, Wachtberg · 2019–2020
BEWiS — Weightless Neural Networks on GPU
Real-time background subtraction that learns what "normal" looks like from the video itself — no training set, no labels.
The industrial problem
On a production line — or any monitored scene — most deviations are visible before they are measurable: a misfeed, a spill, an object where nothing should be. Keeping human eyes on every camera feed does not scale, and threshold sensors miss exactly the unstructured events that matter. What you want is a system that watches a feed, works out on its own what that scene normally looks like, and flags whatever does not belong.
Doing that in real time is the difficulty. The scene is never static: light shifts, the camera vibrates, machinery moves on a cycle, and the "background" itself changes over a shift. Conventional deep learning is the wrong tool here — there is no labelled dataset for a camera that was installed yesterday. Weightless neural networks learn the scene online from raw pixels, but the classical implementation is far too slow to keep up with a live feed.
The system
A second run, decomposed into its three parts. The network was never shown a training set — the background model on the right was assembled from that video itself, frame by frame, while it ran.
[1] Input — the raw camera feed, unmodified.
[2] Foreground — everything the network considers "not normal for this scene".
[3] Background model — the scene the network believes it is looking at.
Scroll the figure sideways →
Inside the system
Figures from the thesis defence — the architecture, the rule that decides what counts as background, and the profile that motivated the rewrite.
The architecture, and why it can learn a scene live. A pixel neighbourhood is binarised, a fixed random mapping addresses a set of RAM neurons, and "recognition" is a table read with a summed response — training is a write, not a gradient step.
Scroll the figure sideways →
How the model decides what counts as "background". Each pixel keeps a short queue of recently seen colours with a stability score; only a colour that stays stable long enough is allowed to train the network — which is what stops a passing truck from being learned as scenery.
Scroll the figure sideways →
Where the time actually goes on a GPU. The compute itself is a small fraction of the budget — allocation and the two host↔device transfers dominate, which is the reason the rewrite targeted memory traffic rather than raw arithmetic.
Scroll the figure sideways →
How it was built
- 01
RAM-based neurons instead of weights
A WiSARD-style weightless network stores what it has seen in lookup tables rather than learned weights: pixels are thermometer-encoded, a random retina mapping wires them to RAM neurons, and recognition is a table read. Training is one write per event — which is why it can learn a scene live.
- 02
Vectorise the algorithm, not just the code
The original implementation punished every memory cell on every frame — 29 million cell touches per frame. Reformulating the decay as something reconstructed at read time (store a counter and a per-cell timestamp, compute the decayed value on access) removed an entire factor from the complexity: a frame now touches only the cells it addresses.
- 03
Prove equivalence at every step
Every optimisation is guarded by a test suite including a cell-exact oracle of the original C++ reference and a dense-versus-lazy engine equivalence test. An optimisation that cannot prove it computes the same thing is not an optimisation — it is a bug with good PR.
What it measured
- 4.3 → 6.8 FPS
- Single CPU thread at 240×320, after the algorithmic rewrite
- ~14×
- Faster than the thesis’ Numba CPU implementation, before any GPU
- 14 / 14
- Tests passing, incl. the C++-semantics oracle
Benchmarks measured on one machine, single-threaded, and documented step by step with the profile that motivated each change (BEWiS-GPU/docs/07_expert_performance.md).
What this builds on
- M.Sc. thesis, Bonn-Aachen International Center for Information Technology (b-it) / Fraunhofer FKIE, 2019
- Implementation of weightless neural networks on GPU for background subtraction — the thesis left "a better GPU implementation of BEWiS" as future work. This package is that implementation.
Archive: Bewis/BEWiS-GPU