Trjtdtk

I am trying to cluster geographical locations in such a way that all the locations inside each cluster are at max within 25 miles of each other. For this, I am using Agglomerative clustering. I am using a custom distance function to calculate the distances between each location. I do not want to specify the number of clusters. Instead, I want the model to cluster until all the locations within each cluster are within 25 miles of each other. I have tried doing this in both Scipy and Sklearn but haven't made any progress. Below is the approach that I have tried. It only gives me one cluster. Please help. Thanks in advance.

from scipy.cluster.hierarchy import fclusterdata 
max_dist = 25
# dist is a custom function that calculates the distance (in miles) between two locations using the geographical coordinates

fclusterdata(locations_in_RI[['Latitude', 'Longitude']].values, t=max_dist, metric=dist, criterion='distance')

I think for HAC (Hierachical Aglomeritive Clustering) it's always helpful to obtain the linkage matrix first which can give you some insight on how the clusters are formed iteratively. Besides that scipy also provides a dendrogram method for you to visualize the cluster formation, which can help you avoid treating the clustering process as a "black box".

import matplotlib.pyplot as plt
from scipy.cluster.hierarchy import dendrogram, linkage

# generate the linkage matrix
X = locations_in_RI[['Latitude', 'Longitude']].values
Z = linkage(X,
 method='complete', # dissimilarity metric: max distance across all pairs of 
 # records between two clusters
 metric='euclidean'
 ) # you can peek into the Z matrix to see how clusters are 
 # merged at each iteration of the algorithm

# calculate full dendrogram and visualize it
plt.figure(figsize=(30, 10))
dendrogram(Z)
plt.show()

# retrive clusters with `max_d`
from scipy.cluster.hierarchy import fcluster
max_d = 25 # I assume that your `Latitude` and `Longitude` columns are both in 
 # units of miles
clusters = fcluster(Z, max_d, criterion='distance')

The clusters is an array of cluster ids, which is what you want.

There is a very helpful (yet kinda long) post on HAC worth reading.