使用 python 进行经纬度聚类

Question

I have a dataset with longitude and latitude information and I need a way to cluster my data if the distance between observations is less than 300m. Anyone has any idea? I tried this:

import pandas as pd
mydata=pd.read_csv("C:\\Users\\Gooljarsd\\Downloads\\restaurantes.csv")
bogota=mydata[(mydata['CITY']=="Bogota")]

import matplotlib.pyplot as plt

from scipy.cluster.hierarchy import dendrogram, linkage

X=bogota[['LAT','LNG']].values
print(X)

Z = linkage(X,
            method='ward',
            metric='euclidean'
    ) 

but I got this error:

MemoryError: Unable to allocate 4.56 GiB for an array with shape (611852671,) and data type float64

Answer 1

I would create a KDTree and then return the indices of points within a certain distance and it should definitely help with your memory issues:

from sklearn.neighbors import KDTree

import numpy as np
import pandas as pd

tree_data = my_data[['LAT', 'LNG']]
tree = KDTree(tree_data, metric='haversine')

bogota = my_data.loc[(my_data.CITY == 'Bogota'), ['LAT', 'LNG']]
idx, dist = tree.query_radius(X=bogota, r=300, return_distance=True)

There are plenty of other methods you can use, for example query() will return n nearest neighbours, so it will be essentially a KNN but with less memory use.

EDIT: Since your data is in lat and long form, the standard Euclidean metric won't work and Haversine metric should be used instead. See other valid metrics here: enter link description here