anomaly detection

Anomaly detection flags the data points that don't look like the rest, and the central engineering constraint is that this almost always has to be learned without labeled examples of what "anomalous" looks like — a fraud team can't enumerate every future fraud pattern in advance, and a factory can't collect labeled images of every possible defect before it happens for the first time. Nearly every method in the field is therefore built the same way: model what normal looks like, then flag whatever doesn't fit that model, rather than training a classifier on both classes the way ordinary supervised learning would.

Isolation Forest flips the usual density-estimation logic and asks how easy a point is to isolate instead. Liu, Ting, and Zhou (2008) built an ensemble of random trees that repeatedly split the data on a randomly chosen feature at a randomly chosen value; a normal point, surrounded by many similar points, takes many splits to separate from its neighbors, while an anomalous point, sitting apart from the main cluster, typically gets isolated in just a few splits. Averaging the path length to isolation across many random trees produces an anomaly score directly, with no need to estimate a density function at all — a computational shortcut that scales well to high-dimensional tabular data and is why Isolation Forest remains a standard first choice in production fraud systems.

Local Outlier Factor fixes a blind spot plain distance-based methods have: it only works when "far" is judged relative to local neighborhood density, not a single global scale. Breunig and colleagues (2000) compared each point's local density to the local density of its neighbors rather than using one fixed distance threshold across the entire dataset, which matters because real data often has clusters of very different densities — a point that looks close to its neighbors in a sparse region and a point that looks close to its neighbors in a dense region are not comparable on a single global distance scale. A point in a sparse neighborhood surrounded by points from a much denser neighborhood gets a high LOF score even if its absolute distances aren't unusually large, which is exactly the case a single global threshold would miss.

Reconstruction-based deep learning methods turn "doesn't fit the normal pattern" into a measurable error. An autoencoder trained only on normal data learns to compress and reconstruct the patterns it was shown, and because it never saw anomalous examples during training, it reconstructs them poorly — the reconstruction error itself becomes the anomaly score, with no separate anomaly-detection logic bolted on. Deep SVDD (Ruff and colleagues, 2018) takes a related but distinct approach: rather than reconstructing the input, it trains a neural network to map all normal data into a tight hypersphere in representation space, and an anomaly is simply a point the network maps outside that sphere — a deep-learning generalization of the classical One-Class SVM, which does the same thing with a kernel method instead of a learned representation.

PatchCore showed that for visual anomaly detection, a model doesn't need to be trained on the target domain's normal data at all — only to have a good enough pretrained feature space. Roth and colleagues (2022) extract features from an ImageNet-pretrained network at multiple spatial scales, build a memory bank of those features computed only from genuinely normal training images, and at inference score a new image by its nearest-neighbor distance in feature space to anything in that memory bank. The approach posted state-of-the-art results on MVTec AD (Bergmann and colleagues, 2019), the standard industrial-defect benchmark with real defective and defect-free photos across fifteen object and texture categories, and it did so with no domain-specific training step at all — just a frozen pretrained backbone and a nearest-neighbor lookup.

Evaluation has to account for the fact that anomalies are rare almost by definition, which makes plain accuracy worthless. If anomalies are one in ten thousand observations, a model that predicts "normal" for everything scores 99.99 percent accuracy while catching zero real anomalies — which is why the field reports precision-recall curves and AUROC rather than accuracy, and why threshold choice is treated as a business decision about acceptable false-positive rate rather than a fixed property of the model. The other persistent problem is concept drift: what counts as "normal" changes over time — seasonal shopping patterns, a new product line, a software update that shifts baseline network traffic — and a model trained once on a snapshot of normal behavior degrades as that baseline moves, which is why production anomaly detection is a continuously retrained system rather than a model trained once and left alone.

Read anomaly detection as model-the-majority-then-flag-the-rest rather than classify-both-classes: every major method, from Isolation Forest's random splits to PatchCore's pretrained feature bank, exists because labeled anomaly examples are structurally scarce, and the entire field is built around squeezing a usable detector out of normal data alone.