data

Data is the raw material every machine learning system is built from — not yet a feature, not yet a label, just the unprocessed record of something that happened: a pixel value, a sensor reading, a line of text, a logged transaction. Everything downstream (features, labels, training sets, models) is a transformation applied to data, which is why data quality limits every model's accuracy more fundamentally than any architecture choice can compensate for — a perfect algorithm trained on bad data still produces a bad model. At a high level, data's journey from raw to useful looks like this:

Structured, unstructured, and semi-structured data each demand a different processing approach, and conflating them is a common source of wasted engineering effort. Structured data (rows and columns in a database, sensor readings on a fixed schedule) is already numeric and tabular, often ready for classical machine learning with minimal transformation; unstructured data (raw text, images, audio, video) carries no inherent numeric structure and needs a feature-extraction step — tokenization for text, pixel-to-tensor conversion for images — before any model can touch it; semi-structured data (JSON logs, XML documents) sits in between, with some explicit structure but fields that vary record to record. Roughly 80-90% of the data generated by most organizations is unstructured, which is the direct reason deep learning's ability to automatically extract features from raw unstructured input, rather than needing a human to hand-engineer structure first, was such an economically significant shift.

Data volume growth is itself a direct driver of chip and storage demand, independent of any specific model being trained on it. IDC estimates put global data creation in the hundreds of zettabytes annually by the mid-2020s, and that raw volume has to be stored, moved, and processed before any of it reaches a training run — this is the reason data infrastructure (high-bandwidth storage, fast interconnects between storage and compute, data-center networking) is as real a bottleneck in large-scale machine learning as the GPUs themselves, since a GPU cluster idling while waiting for data to stream in from storage is wasted compute regardless of how fast the chips themselves are.

Data quality problems compound invisibly, which is why "garbage in, garbage out" understates the real risk rather than overstating it. Mislabeled examples, duplicate records, and biased sampling don't just add noise that averages out — they actively teach a model incorrect patterns that look statistically valid within the flawed dataset, which is exactly how a model can achieve high accuracy on its own training and validation data while still being wrong about the real world it's meant to operate in. This is why data engineering and data cleaning, unglamorous as they are relative to model architecture work, consistently turn out to be where the actual accuracy gains in a production system come from.

Read data as the one input to machine learning that no architecture, however clever, can fully compensate for: a model is a lossy compression of whatever data it was shown, and the shape, volume, and quality of that data — not the choice of algorithm — is what sets the ceiling on what any model trained on it can ever actually learn.