12 Projects
12.1 Image Clustering and Compression
The goal of this analysis is to simplify an image by reducing the number of distinct colors while preserving its overall appearance.
In other words, we are using clustering to group similar pixels in the RGB color space so that all pixels within a cluster share a representative color (the centroid or medoid).
This technique:
- Reduces file size (useful for compression),
- Highlights dominant colors (helpful in computer vision, design, and pattern recognition),
- Illustrates how clustering groups similar observations — here, similar colors.
So, clustering acts like a smart palette optimizer: it finds the key “color families” that best describe the image.
12.1.1 Load and Visualize the Image
Here, we use a small flower image.
You can replace the URL with any image (small then 1 MB).
Before applying clustering, let us look at what our image data actually looks like.
When we read the image, each pixel is represented by three values — Red (R), Green (G), and Blue (B) — that together define its color. So, an image becomes a dataset where each row = one pixel, and the columns are R, G, and B. This is why we can treat image processing as a clustering problem — we want to group pixels with similar colors.
Now that we understand what the data looks like (each pixel is a point in RGB space), our next goal is to group pixels with similar colors into clusters.
We will start with \(k\)-means and later, will do \(k\)-medoids. At the end, we will compare both methods.
12.1.2 \(k\)-means
12.1.2.1 Choosing the Number of clusters
We already know three standard methods for determining the best \(k\):
- Elbow Method: looks at how the within-cluster sum of squares (WSS) decreases with increasing \(k\); the elbow marks a good balance between simplicity and accuracy.
- Silhouette Coefficient: evaluates how well each point fits within its cluster; the closer to \(1\), the better.
- Gap Statistic: compares within-cluster dispersion to that of random uniform data.
We use all three to guide our choice of \(k\).
Because clustering every pixel in a large image is computationally heavy and memory-intensive, we can analyze only a random subset of pixels that still represents the overall color diversity.
In clustering, what matters most is diversity in the data, not volume. A well-chosen random sample gives us the same structure at a fraction of the cost. Once we know how many clusters we need, we can apply that to all pixels for the final image.
If our image has many similar pixels (which most real images do), then sampling \(5\,000\) pixels is enough to capture all the dominant color regions. If the image has tiny regions of rare colors (e.g., coral reef photos, satellite imagery), a small sample might miss some color groups. In this case, use a larger sample (e.g., \(20\,000\)).
Here, we use a sampled dataset (sample_img) to decide the optimal \(k\).
Based on this plot, we can see that WSS decreases sharply until around \(k = 3\), after which the improvement flattens. So, the “elbow” suggests that \(3\) clusters capture most of the color variance efficiently.
The average silhouette width peaks at \(k = 2\), meaning that clusters are most compact and well-separated at this point. However, the drop from \(k = 2\) to \(k = 3\) is small, and with images (continuous color gradients), slightly lower silhouette scores are expected for more nuanced color palettes.
The gap statistic reaches its maximum around \(k = 4\), meaning that \(4\) clusters explain the data structure significantly better than a random uniform reference.
What do we choose?
When methods disagree slightly, Elbow: 3 clusters, Silhouette: 2 clusters, Gap-Statistic: 4 clusters, a reasonable consensus is to choose \(k = 3\) or \(k = 4\), depending on how much color detail we want in the compressed image. Choosing \(k = 3\) gives smoother and more general color areas. And, \(k = 4\) preserves finer details that is more realistic for natural photos.
In the following code, we fit two \(k\)-means models and reconstruct the compressed images and show them side-by-side to see how the number of clusters affects color richness and smoothness.
In the original image, each pixel’s RGB value is treated as a 3D point in color space. In this output, each pixel is replaced by its cluster’s centroid color.
The image based on \(k = 3\) in the left shows an image that is compressed into just 3 colors and it looks a poster with strong color bands and high compression. In the right image (i.e., choosing \(k = 4\)), a bit more detail and smoother gradients appear, especially in the faces and background, but still simplified compared to the original.
Increasing \(k\) adds more colors which results in higher visual quality but less compression.
At what value of k does the image start to look realistic again?
As we see, at \(k=3\), the compression is extreme – only broad regions of color remain, and fine details (like skin tone, clothing patterns, or light gradients) are lost. Around \(k = 6-8\), the image starts to look visually realistic again, as the model captures enough distinct color shades to represent lighting, texture, and contrast. Beyond that, improvements become barely noticeable, while file size and computation cost increase.
12.1.3 \(k\)-Medoids (PAM)
While \(k\)-means uses centroids (means of points in a luster), \(k\)-medoids uses medoids (actual data points that minimize total dissmilarity). This makes \(k\)-medoids to be more robust to outliers, works with non-Euclidean distances, although it is more computationally expensive.
In this example, each pixel is treated as a point in RGB space again but instead of averaging colors (which can produce unrealistic shades), \(k\)-medoids selects real pixel colors as representives.
The Gap Statistic is computationally heavy because it repeatedly compares real clustering quality to that of random data. For large datasets like images, we either reduce the bootstrap count or rely on faster methods such as the Silhouette coefficient.
The results for \(k\)-medoids (PAM) show consistent patterns with those obtained from \(k\)-means. Elbow methods suggests 3 clusters. Silhouette method indicates 2 clusters as the configuration with the most compact and well-separated groups. Gap statistic peaks at 4 clusters.
Although the exact optimal \(k\) differs slightly between methods, all three converge on the same range (2-4 clusters), confirming that the image’s dominant colors can be represented effectively with only a few color groups. This agreement also shows that \(k\)-medoids captures a similar underlying structure as \(k\)-means, while being more robust to outliers and based on actual data points rather than centroids; so, you should expect more time for seeing the output.
To better understand how the number of clusters affects the representation, we wil test \(k = 3\) and \(k = 4\) with the PAM algorithm.
Unlike \(k\)-means (which assumes Euclidean distance), PAM allows any dissimilarity measure.
For image data (and in general, for colour data), Euclidean usually works well, but we can test Manhattan as well, since it can handle abrupt colour transitions more robustly.