Structure from Noise

Machine learning systems excel at finding patterns in chaos, extracting meaningful knowledge from what appears to be noise.

Finding Businesses in a Haystack of Google Usernames

During my time at Google, where I worked on spam and abuse prevention on Google Maps, one of the smaller projects was to identify Gmail accounts of businesses vs regular users. The challenge: usernames look like random character strings.

But hidden patterns exist. Coffee shops create “hello.thirdwave” or “contact_singleorigin”. Regular users choose “winston.smith1984” or “coffeelover42”.

Our approach was simple: character n-gram models. Breaking usernames into overlapping sequences of 3-5 characters, we trained a linear model to learn which patterns signaled businesses versus personal accounts.

The model discovered distinct patterns:

  • Business-leaning grams: “inf”, “nfo”, “sup”, “por”, “sal”, “les”, “hel”, “llo”, “ltd”, “inc”
  • Personal-leaning grams: name fragments like “ann”, “jon”, “smi”, “ith”, plus birthday patterns “198”, “984”, “200”, “001”

No hand-written rules. Just weights learned from labeled examples. This simple approach worked nearly as well as much more complex models (pre-trained embeddings, custom architectures) while being considerably easier to maintain.

Here’s a demonstration of the core idea:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline

pipe = Pipeline([
    ("tfidf", TfidfVectorizer(analyzer="char", ngram_range=(3,5), min_df=5)),  # min_df: ignore rare n-grams
    ("clf", LogisticRegression(max_iter=100, class_weight={0: 1, 1: 20}))  # ~5% are businesses
])
pipe.fit(usernames_train, labels_train)
print(f"Accuracy: {pipe.score(usernames_test, labels_test):.2%}")

The precision was remarkable—better than 90% on internal evaluations from nothing but character sequences.

The Netflix Magic

Collaborative filtering demonstrates the same principle. From a sparse matrix of user-item ratings, it learns latent neighborhoods: users who loved “The Matrix” also rated “Equilibrium” highly.

Technically, it approximates the rating matrix $R$ with low-rank factors $U \cdot V^T$. Proximity in that latent space predicts taste. Structure emerges from co-occurrence patterns alone, no genre tags or plot summaries needed.

The Principle

Statistical patterns alone can distinguish businesses from personal accounts, predict movie tastes, and solve countless other problems. Why does this work?

  • Compressibility = Predictability: If data has recurring patterns, a model can compress it (few features explain many cases) and predict well
  • Local shards, global signal: Tiny n-grams or pairwise co-occurrences, aggregated at scale, reveal global structure
  • No rules required: The model assigns weights; the data supplies the “rules”

Whether finding businesses in username chaos or movie preferences in rating matrices, the principle remains: feed the algorithm enough apparent noise, and it will find the hidden structure within.