如何在python中从Lat和Long获得带有关联邮政编码的Dataframe？

Question

I am trying to reverse Geocode and get the zip codes. I have a table / dataframe that has 400,000 lat and long and I would like to get the zip code by parsing the lat,long.

Below is the sample dataframe

data = {'Site 1': '31.336968, -109.560959',
        'Site 2': '31.347745, -108.229963',
        'Site 3': '32.277621, -107.734724',
        'Site 4': '31.655494, -106.420484',
        'Site 5': '30.295053, -104.014528'}

My code:

import geopandas as gpd
from shapely.geometry import Point
gdf_locations = gpd.read_file('/Users/admin/Downloads/tl_2016_us_zcta510/tl_2016_us_zcta510.shp')

I downloaded the tl_2016_us_zcta510.shp file from here.

This is where I am getting stuck. One solution that I tried was to convert it to NUMPY array and pass the value. But that seems extremely slow. I would like to do it as a dataframe using lambda and get the results quickly.

What I tried:

               [Longitude]    [Latitude]
x = np.array((-73.986946106, 40.284328461))
x_pnt = Point(x)
filter = gdf_locations['geometry'].contains(x_pnt)
print(gdf_locations.loc[filter, 'GEOID10'])

While this is giving me what I want, it is extremely slow. How can I make it faster and as a recursive function? Any help is appreciated. Thank you.

PS: I have seen many blog posts and read stuff on this subject, but none seems to address it for large scale real-time implementation.

Edits: I am specifically looking to get a dataframe with following structure:

 data = {'Site 1': '31.336968, -109.560959', 94108,
            'Site 2': '31.347745, -108.229963', 60616,
            'Site 3': '32.277621, -107.734724', 78654,
            'Site 4': '31.655494, -106.420484', 78090,
            'Site 5': '30.295053, -104.014528', 78901}

I understand how to convert lat long to Zip, what I'm not able to do is get a dataframe. Hope this makes it more clear.

Answer 1

I haven't used geopandas very much, but I would try using scipy's cKDTree . It should be very fast for the amount of data you have. The only thing is that it works for point to point lookups, so you'd have to use the centroids of the polygons from the zip code dataset.

For example, transforming the centroids, which are Shapely points, to a numpy array:

centroids = gdf_locations.centroid

# transform shapely points to np array
point_array = []
for centroid in centroids:
    point_array.append([centroid.x, centroid.y])
point_array = np.array(point_array)

print(point_array[0])
>>> array([-83.61511443,  41.31279856])

To make sure it was going to be a quick lookup I create 400,000 random coordinates:

random_lat_long = np.random.randn(400000, 2) * 80

print(random_lat_long)
>>> array([ -8.37429385, -23.19458311])

Now for the closest point:

distance, index = spatial.cKDTree(point_array).query(random_lat_long)

On my computer using %%timeit in Jupyter this took about 1.7 seconds.

And finally, grabbing the zip codes from the dataframe:

zip_codes = gdf_locations.loc[index, 'GEOID10']

Edit: To get the latitude and longitude as part of the result:

Pull out data and convert types:

lats_lons_zips = gdf_locations.loc[index, ['INTPTLAT10', 'INTPTLON10', 'GEOID10']]
# keep the zip code as an str to preserve leading zeros
lats_lons_zips = lats_lons_zips.astype({"INTPTLAT10": float, "INTPTLON10": float, "GEOID10": str})

Change index to "Site XX":

new_index = ["Site " + str(i) for i in range(len(lats_lons_zips))]
lats_lons_zips.index = new_index

Finally, get results:

print(lats_lons_zips.iloc[0:4].to_dict(orient="index"))

{ 'Site 0': { 'GEOID10': '00824',
              'INTPTLAT10': 17.7445568,
              'INTPTLON10': -64.6829328},
  'Site 1': { 'GEOID10': '96916',
              'INTPTLAT10': 13.2603723,
              'INTPTLON10': 144.7006789},
  'Site 2': { 'GEOID10': '96916',
              'INTPTLAT10': 13.2603723,
              'INTPTLON10': 144.7006789},
  'Site 3': { 'GEOID10': '04741',
              'INTPTLAT10': 47.453712,
              'INTPTLON10': -69.2229208}}

Answer 2

Maybe you need geopandas.sjoin .

In a Spatial Join, two geometry objects are merged based on their spatial relationship to one another.

First you need to prepare the site data to geoDataFrame.

import geopandas as gpd
import pandas as pd
from shapely.geometry import Point

gdf_locations = gpd.read_file('tempdata/tl_2016_us_zcta510.shp')
data = {'Site 1': '31.336968, -109.560959',
        'Site 2': '31.347745, -108.229963',
        'Site 3': '32.277621, -107.734724',
        'Site 4': '31.655494, -106.420484',
        'Site 5': '30.295053, -104.014528'}
df_site = pd.DataFrame.from_dict(data, orient='index',columns=['locstr'])
df_site['loc'] = df_site['locstr'].apply(lambda x: list(map(float,x.split(','))))
df_site['loc'] = df_site['loc'].apply(lambda x: Point(x[1],x[0]))
gdf_site = gpd.GeoDataFrame(df_site,geometry=df_site['loc'],crs=gdf_locations.crs).drop(['loc'], axis=1)
print(gdf_site)

                        locstr                       geometry
Site 1  31.336968, -109.560959  POINT (-109.560959 31.336968)
Site 2  31.347745, -108.229963  POINT (-108.229963 31.347745)
Site 3  32.277621, -107.734724  POINT (-107.734724 32.277621)
Site 4  31.655494, -106.420484  POINT (-106.420484 31.655494)
Site 5  30.295053, -104.014528  POINT (-104.014528 30.295053)

Then you can set op='intersects' to judge the Spatial Relation between site and shp.

intersects: The attributes will be joined if the boundary and interior of the object intersect in any way with the boundary and/or interior of the other object.

gdf_site = gpd.sjoin(gdf_site,gdf_locations,how='left',op='within')
print(gdf_site[['locstr','GEOID10']])

                        locstr GEOID10
Site 1  31.336968, -109.560959   85607
Site 2  31.347745, -108.229963   88040
Site 3  32.277621, -107.734724   88030
Site 4  31.655494, -106.420484     NaN
Site 5  30.295053, -104.014528   79843