Hierarchical Clustering: Groups Inside Bigger Groups
Sometimes one flat list of groups is not enough
Support tickets can have broad themes such as payment and delivery, then smaller themes such as payment pending and duplicate charge. You may want to see how small groups join into larger ones rather than choosing one K immediately.
Hierarchical clustering creates a nested grouping. The common bottom-up version, called agglomerative clustering, starts with each row alone and repeatedly merges the closest groups.
Follow four points by hand
Take A = 1, B = 2, C = 8, and D = 10 on a single numeric axis. A and B are 1 unit apart. C and D are 2 units apart. With single linkage, AB and CD eventually join across their nearest pair, B = 2 and C = 8, at distance 6.
Single linkage: merge at distances 1, 2, then 6
The green cut at distance 3 gives two groups: AB and CD. Horizontal leaf spacing is for layout only.
1points = {"A": 1, "B": 2, "C": 8, "D": 10}
2print("A-B distance:", abs(points["A"] - points["B"]))
3print("C-D distance:", abs(points["C"] - points["D"]))
4left, right = [1, 2], [8, 10]
5print("AB-CD single linkage:", min(abs(a - b) for a in left for b in right))A-B distance: 1 C-D distance: 2 AB-CD single linkage: 6
Distance between groups needs a rule
Distance between two individual points is straightforward. Distance between two sets of points needs a choice. That choice is called linkage, and it can change the entire hierarchy.
Key Points
- Single linkage uses the closest pair across the groups. It can join distant regions through a chain of intermediate points.
- Complete linkage uses the farthest pair. It favours groups whose members remain reasonably close to each other.
- Average linkage averages all pair distances across the two groups.
- Ward linkage chooses the merge with the smallest increase in within-cluster squared variation. It requires Euclidean distance in the usual implementation.
Single linkage sees a bridge; complete linkage sees the widest separation. Neither is automatically correct. Choose according to what a coherent group means in your problem.
Compare linkage values for the same groups
1left, right = [1, 2], [8, 10]
2distances = [abs(a - b) for a in left for b in right]
3print("Pair distances:", distances)
4print("Single:", min(distances))
5print("Complete:", max(distances))
6print(f"Average: {sum(distances) / len(distances):.1f}")Pair distances: [7, 9, 6, 8] Single: 6 Complete: 9 Average: 7.5
The input rows did not change. Only the definition of distance between groups changed. This is why a hierarchy should always be reported with its feature representation, distance, and linkage.
Read a dendrogram from the bottom upward
A dendrogram is the tree drawing of merges. Individual rows begin at the bottom. Lines join at the height where their groups merge. A horizontal cut through the tree leaves one cluster for each branch it crosses.
In the four-point example, a cut above distance 2 but below 6 gives AB and CD. A cut below 1 leaves four individual rows. The left-to-right order of leaves is not a measurement of similarity; merge heights and connections carry the meaning.
Tip:A dendrogram is a grouping diagram, not an evolutionary family tree. Shared branches do not prove that one customer type developed from another.
Fit two clusters with scikit-learn
1from sklearn.cluster import AgglomerativeClustering
2
3values = [1, 2, 8, 10]
4X = [[value] for value in values]
5model = AgglomerativeClustering(n_clusters=2, linkage="single")
6labels = model.fit_predict(X)
7groups = sorted(
8 [values[i] for i in range(len(values)) if labels[i] == label]
9 for label in set(labels)
10)
11print("Groups:", groups)Groups: [[1, 2], [8, 10]]
We sort member lists for display because cluster IDs have no inherent order. An alternative configuration selects clusters by a distance threshold rather than a requested count.
A merge is usually not undone
Agglomerative methods greedily choose the next merge. Once two groups merge, the standard method does not split them again. Early decisions can therefore influence later groups.
Key Points
- Scale features when units should not dominate.
- Compare plausible linkage choices.
- Inspect sensitivity to outliers or bridge points.
- Use labels or domain review only as evaluation evidence, not as invented proof that the hierarchy is natural.
Large datasets and new rows need extra thought
Many hierarchical methods need substantial pairwise distance work or storage, often growing roughly with the square of the number of rows. A plot of thousands of leaf names also becomes difficult to read.
scikit-learn’s AgglomerativeClustering does not offer a normal predict method for new rows. Refit on a new dataset or design a separate assignment policy and validate it. A nearest-centre policy would be your added rule, not something the hierarchy originally learned.
Where people get confused
Key Points
- Not choosing K at the start does not eliminate the grouping decision. You still choose a cut or cluster count to use the hierarchy.
- Leaf order is not distance. Look at merge connections and heights.
- Different linkage rules can create different valid trees from the same rows.
- A nested pattern is not proof of ancestry, causation, or a universal customer taxonomy.
Quick check
How does agglomerative clustering begin?
Each row starts as its own group, then the method repeatedly merges the closest groups under its distance and linkage rules.
Why is linkage needed?
A group contains several points, so we need a definition of distance between groups. Closest-pair, farthest-pair, average, and Ward rules differ.
What does cutting a dendrogram do?
A cut at a chosen height selects the groups whose branches remain separate below that height.
Why can a new-row assignment need a separate policy?
The fitted hierarchy describes merges among the fitting rows. AgglomerativeClustering does not provide ordinary out-of-sample predict, so a new assignment rule must be designed or the method refitted.
Keep the hierarchy honest
Key Points
- Report distance, scaling, linkage, and the chosen cut.
- Inspect group members and merge heights together.
- Be careful with chains, outliers, and quadratic-scale work.
- Next, DBSCAN finds dense regions and can leave isolated rows as noise.
Theia
Can you explain this concept?
Practise Hierarchical Clustering: Groups Inside Bigger Groups with realistic interview questions. Choose spoken or typed answers, get feedback on your reasoning, then return to the material that needs work.
Sign in to practise this conceptThe session stays focused on this lesson. Your selected interview level sets the bar.