No labels, still a pattern.

Lesson 6 of 7About 9 minutes5 steps

Unsupervised learning finds structure with no target to predict: clustering groups similar customers, and PCA squeezes many columns into a few. Both are easy to run and easy to misread.

  • Explain k-means and how to choose k
  • Say why scaling matters
  • Say what PCA keeps, and when it can’t help

You need: Lesson 3.

  1. 1Distributions and sampling
  2. 2Confidence intervals and tests
  3. 3Training, testing and overfitting
  4. 4Judging a classifier
  5. 5Trees and ensembles
  6. 6Clustering and PCA
  7. 7How a neural network learns
0 of 5 steps

  1. Groups without labels

    Predict first

    k-means groups the churn customers by tenure and complaints into 3, never seeing who left. Do the groups differ in churn?

    Just show me

    Yes: one group leaves at 2.3%, another at 12.4%, though churn was never shown.

    Groups built from behaviour often line up with outcomes. Here: long-standing customers with few complaints (about 58 months on average), newer ones with no complaints (about 20 months) and the customers who complain (about 1.3 complaints each in 90 days).

    Long-standing, few complaints (1,840)2.3%
    Newer, no complaints (1,755)7.4%
    Customers who complain (1,405)12.4%

    Share who left in each k-means group (k = 3; tenure and complaints, scaled), scikit-learn 1.8.0

    Quick check

    What does k-means need you to choose?

    k is set in advance; the elbow and the silhouette score help choose it.

  2. How k-means works

    Predict first

    k-means starts with 3 random centres. What does it repeat?

    Just show me

    Assign each point to its nearest centre, move the centres to their groups’ averages, and repeat until nothing changes.

    It shrinks the distances from points to their group’s centre, so it finds round groups of similar spread, and different starting centres can end differently, which is why scikit-learn runs it several times (n_init) and keeps the best. Choose k with the elbow in those distances, the silhouette score, and whether the groups mean something to the business.

    Quick check

    Your groups are long and curved, like crescents. Is k-means a good fit?

    k-means assumes round groups; density-based methods follow other shapes.

  3. Scale first

    Predict first

    Cluster on tenure (1 to 72 months) and complaints (0 to 4) without scaling. What dominates?

    Just show me

    Tenure: distances in months swamp distances in complaints, so the groups would all but ignore complaints.

    k-means, KNN, PCA and SVMs all measure distances, so a column in bigger units counts for more. Standardising each column to a mean of 0 and a spread of 1, as here, gives each the same say. Trees don’t care about scale.

    Quick check

    Which model doesn’t need scaled features?

    Trees split one column at a time at a threshold, so its units don’t matter.

  4. PCA: fewer columns, most of the information

    Predict first

    PCA on the table’s three numeric columns. How much of the variation does the first component capture?

    Just show me

    34%: the three components share it almost equally (34%, 34%, 32%).

    PCA finds new axes, mixtures of the columns, ordered by how much variation each captures. It compresses well only when columns move together. The table’s code draws these three independently, so there’s no shared direction to find, and dropping any component loses a third. On real data with related columns (sizes, prices, sensor readings), a few components often hold most of it.

    Component 134.3%
    Component 233.9%
    Component 331.8%

    Share of the variation each principal component captures (tenure, monthly charge and complaints, scaled)

    Quick check

    When does PCA help most?

    Correlated columns share directions, which a few components can capture.

  5. Judge them with care

    Predict first

    You found 4 groups of customers. How do you judge them?

    Just show me

    There’s no label to score against: judge by the silhouette, stability across samples, and usefulness.

    Check that the groups hold on a fresh sample, describe each in a sentence a marketer would use, and test any decision based on them, ideally in an A/B test. PCA’s components are hard to name too: each mixes many columns, which suits compression and plots more than explanations.

    Quick check

    Why can PCA make a model harder to explain?

    “Component 2 is high” means little to a manager; “tenure is short” means a lot.

In the interview: clustering and PCA.

Ask Prism about this lesson

Sources

  1. scikit-learn, “Clustering”, k-means, DBSCAN and how to judge clusters.
  2. Jolliffe and Cadima, “Principal component analysis: a review and recent developments”, Philosophical Transactions of the Royal Society A, 2016.
  3. The ML guide’s churn table, k-means and PCA run with scikit-learn 1.8.0.

Back: lesson 5, Trees and ensembles