Trjtdtk

I am trying to perform k-means clustering on multiple columns. My data set is composed of 4 numerical columns and 1 categorical column. I already researched previous questions but the answers are not satisfactory.

I know how to perform the algorithm on two columns, but I'm finding it quite difficult to apply the same algorithm on 4 numerical columns.

I am not really interested in visualizing the data for now, but in having the clusters displayed in the table.The picture shows that the first row belongs to cluster number 2, and so on. That is exactly what I need to achieve, but using 4 numerical columns, therefore each row must belong to a certain cluster.

Do you have any idea on how to achieve this? Any idea would be of great help. Thanks in advance! :

There is no difference in methodology between 2 and 4 columns. If you have issues then they are probably due to the contents of your columns. K-Means wants numerical columns, with no null/infinite values and avoid categorical data. Here I do it with 4 numerical features:

import pandas as pd
from sklearn.datasets.samples_generator import make_blobs
from sklearn.cluster import KMeans

X, _ = make_blobs(n_samples=10, centers=3, n_features=4)

df = pd.DataFrame(X, columns=['Feat_1', 'Feat_2', 'Feat_3', 'Feat_4'])

kmeans = KMeans(n_clusters=3)

y = kmeans.fit_predict(df[['Feat_1', 'Feat_2', 'Feat_3', 'Feat_4']])

df['Cluster'] = y

print(df.head())

Which outputs:

 Feat_1 Feat_2 Feat_3 Feat_4 Cluster
0 0.005875 4.387241 -1.093308 8.213623 2
1 8.763603 -2.769244 4.581705 1.355389 1
2 -0.296613 4.120262 -1.635583 7.533157 2
3 -1.576720 4.957406 2.919704 0.155499 0
4 2.470349 4.098629 2.368335 0.043568 0

Let's take as an example the Breast Cancer Dataset from the UCI Machine Learning.

This is how it looks

>> _data.head(5)

 Age BMI Glucose Insulin HOMA Leptin Adiponectin Resistin 
0 48 23.500000 70 2.707 0.467409 8.8071 9.702400 7.99585 
1 83 20.690495 92 3.115 0.706897 8.8438 5.429285 4.06405 
2 82 23.124670 91 4.498 1.009651 17.9393 22.432040 9.27715 
3 68 21.367521 77 3.226 0.612725 9.8827 7.169560 12.76600 
4 86 21.111111 92 3.549 0.805386 6.6994 4.819240 10.57635 

 MCP.1 Classification 
0 417.114 1 
1 468.786 1 
2 554.697 1 
3 928.220 1 
4 773.920 1 

As you can see, all the columns are numerical. Let's see now, how we can cluster the dataset with K-Means. We don't need the last column which is the Label.

### Get all the features columns except the class
features = list(_data.columns)[:-2]

### Get the features data
data = _data[features]

Now, perform the actual Clustering, simple as that.

clustering_kmeans = KMeans(n_clusters=2, precompute_distances="auto", n_jobs=-1)
data['clusters'] = clustering_kmeans.fit_predict(data)

There is no difference at all with 2 or more features. I just pass the Dataframe with all my numeric columns.

 Age BMI Glucose Insulin HOMA Leptin Adiponectin Resistin 
0 48 23.500000 70 2.707 0.467409 8.8071 9.702400 7.99585 
1 83 20.690495 92 3.115 0.706897 8.8438 5.429285 4.06405 
2 82 23.124670 91 4.498 1.009651 17.9393 22.432040 9.27715 
3 68 21.367521 77 3.226 0.612725 9.8827 7.169560 12.76600 
4 86 21.111111 92 3.549 0.805386 6.6994 4.819240 10.57635 

 cluster 
0 0 
1 0 
2 0 
3 0 
4 0 

How you can visualize the clustering now? Well, you cannot do it directly if you have more than 3 columns. However, you can apply a Principal Component Analysis to reduce the space in 2 columns and visualize this instead.

### Run PCA on the data and reduce the dimensions in pca_num_components dimensions
reduced_data = PCA(n_components=pca_num_components).fit_transform(data)
results = pd.DataFrame(reduced_data,columns=['pca1','pca2'])

sns.scatterplot(x="pca1", y="pca2", hue=data['clusters'], data=results)
plt.title('K-means Clustering with 2 dimensions')
plt.show()

Here are the imports I used

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from sklearn.cluster import KMeans

And this is the visualization