Find the distance between points and line and achieve the projection distance that the point takes in a line

Question

I am new in python. But I have a challenging task for me: I have CSV files with GPS coordinates and 1 file with reference trajectory (coordinates points) which represent the track of the vehicle.

So the task is to all points calculate the perpendicular distance to reference trajectory and define in which kilometer this point is in the reference trajectory

I was trying to solve it with QGIS but the problem is that it is too much data and QGIS all the time crashed. I was using the distance matrix.

So the reference trajectory data looks like this:

    datetime            lon_deg     lat_deg     ki1ometers  
0   27.03.2018 15:07    14.34559621 48.28282695 0   
1   27.03.2018 15:07    14.34539589 48.283579   0.08492765648897423 
2   27.03.2018 15:08    14.34509878 48.28437137 0.17573647034625345 
3   27.03.2018 15:08    14.34476681 48.28520735 0.2718837851891085  
4   27.03.2018 15:09    14.34440297 48.28607467 0.372012272777317   
5   27.03.2018 15:10    14.34355387 48.28785601 0.5798125761498747  
6   27.03.2018 15:10    14.34312139 48.28876254 0.6855708866782635  
7   27.03.2018 15:11    14.34267986 48.28966368 0.7909635418697577  
8   27.03.2018 15:11    14.34235909 48.29057934 0.895509507334529   
9   27.03.2018 15:12    14.34193015 48.29147634 1.000178064181187   
10  27.03.2018 15:12    14.34158939 48.2923968  1.1055875957864745  
11  27.03.2018 15:13    14.34125444 48.29332421 1.2116463089787737  
12  27.03.2018 15:13    14.34084938 48.29424082 1.31788253222638    
13  27.03.2018 15:14    14.34041673 48.29515665 1.4246295164890292  
14  27.03.2018 15:14    14.34001362 48.29608703 1.532295241219843   
15  27.03.2018 15:15    14.33959522 48.29702238 1.6408091272201002  
16  27.03.2018 15:15    14.33917898 48.29796904 1.7504838454702525  
17  27.03.2018 15:16    14.33875624 48.29892358 1.8611345768980705  
18  27.03.2018 15:16    14.33832484 48.29988211 1.9723928345544686  
19  27.03.2018 15:17    14.337844699999998  48.30083163 2.083788039109954   
20  27.03.2018 15:17    14.33733187 48.30177414 2.1952441083077696  
21  27.03.2018 15:18    14.33680756 48.30271439 2.3067561380904458  
22  27.03.2018 15:18    14.33637327 48.30366977 2.4177398933361665  
23  27.03.2018 15:19    14.33579109 48.30456609 2.5263104564169723  

And the data that I need to calculate distance and position on my reference trajectory:

datetime                lon_deg             lat_deg
2018-01-29 00:00:00.000 13.535165989333333  48.58077572716667
29.01.2018 0:00         13.535166009        48.580775726166664
2018-01-29 00:00:01.000 13.535165977166667  48.580775749
29.01.2018 0:00         13.5351658175       48.58077575
2018-01-29 00:00:02.000 13.535165976833333  48.58077567466667
29.01.2018 0:00         13.535165988166666  48.58077563316667
2018-01-29 00:00:03.000 13.535165978333334  48.580775599
29.01.2018 0:00         13.535166127833334  48.5807756575
2018-01-29 00:00:04.000 13.535166430833334  48.5807757935
29.01.2018 0:00         13.535166510166666  48.580775819
2018-01-29 00:00:05.000 13.5351665845       48.5807758835
29.01.2018 0:00         13.5351665215       48.580775906
2018-01-29 00:00:06.000 13.535166549166666  48.58077594583333
29.01.2018 0:00         13.535166521333334  48.58077594466667
2018-01-29 00:00:07.000 13.535166487        48.580775927666664
29.01.2018 0:00         13.5351670905       48.58077611433333
2018-01-29 00:00:08.000 13.5351669075       48.5807760195
29.01.2018 0:00         13.535166444166666  48.580775919

So the output I expect is calculated distance to the line trajectory and in which specific kilometer of trajectory this point is.

I would be happy to hear any ideas because I really stuck in this problem

Answer 1

In case of such small distances (like a trip of 2.5 km), you can probably live with a planar approximation and use very simple stuff: for the distance you can use formulas for calculating the height of a triangle, like the one working with semiperimeter . Then you will still have to calculate if the point is "in" the given line segment (part of the perpendicular strip built on it), where you can use the projection property of scalar (dot) product of vectors.

So for a line-segment AB, and a point C (all points are tuples/lists) you would need something like this:

pdist=lambda A,B:((A[0]-B[0])**2+(A[1]-B[1])**2)**(1/2)

def dist(A,B,C):
  c=pdist(A,B)
  rat=((C[0]-A[0])*(B[0]-A[0])+(C[1]-A[1])*(B[1]-A[1]))/c/c
  if rat<0 or rat>1:
    return None,None
  a=pdist(B,C)
  b=pdist(A,C)
  s=(a+b+c)/2
  alt=2*(s*(s-a)*(s-b)*(s-c))**(1/2)/c
  return alt,rat

It returns None,None if the perpendicular base point for C is outside AB, or a tuple of perpendicular distance (of C from AB segment) and a ratio between 0...1 which you can use as weight for calculating "tour-position" between the two neighboring points.

Then some parsing on your example data:

import re
rawtour='''0   27.03.2018 15:07    14.34559621 48.28282695 0   
1   27.03.2018 15:07    14.34539589 48.283579   0.08492765648897423 
2   27.03.2018 15:08    14.34509878 48.28437137 0.17573647034625345 
3   27.03.2018 15:08    14.34476681 48.28520735 0.2718837851891085  
4   27.03.2018 15:09    14.34440297 48.28607467 0.372012272777317   
5   27.03.2018 15:10    14.34355387 48.28785601 0.5798125761498747  
6   27.03.2018 15:10    14.34312139 48.28876254 0.6855708866782635  
7   27.03.2018 15:11    14.34267986 48.28966368 0.7909635418697577  
8   27.03.2018 15:11    14.34235909 48.29057934 0.895509507334529   
9   27.03.2018 15:12    14.34193015 48.29147634 1.000178064181187   
10  27.03.2018 15:12    14.34158939 48.2923968  1.1055875957864745  
11  27.03.2018 15:13    14.34125444 48.29332421 1.2116463089787737  
12  27.03.2018 15:13    14.34084938 48.29424082 1.31788253222638    
13  27.03.2018 15:14    14.34041673 48.29515665 1.4246295164890292  
14  27.03.2018 15:14    14.34001362 48.29608703 1.532295241219843   
15  27.03.2018 15:15    14.33959522 48.29702238 1.6408091272201002  
16  27.03.2018 15:15    14.33917898 48.29796904 1.7504838454702525  
17  27.03.2018 15:16    14.33875624 48.29892358 1.8611345768980705  
18  27.03.2018 15:16    14.33832484 48.29988211 1.9723928345544686  
19  27.03.2018 15:17    14.337844699999998  48.30083163 2.083788039109954   
20  27.03.2018 15:17    14.33733187 48.30177414 2.1952441083077696  
21  27.03.2018 15:18    14.33680756 48.30271439 2.3067561380904458  
22  27.03.2018 15:18    14.33637327 48.30366977 2.4177398933361665  
23  27.03.2018 15:19    14.33579109 48.30456609 2.5263104564169723  '''
tour=list(map(lambda line:list(map(lambda x:float(x),re.match(r'[^\s]+\s+[^\s]+\s+[^\s]+\s+([\d\.]+)\s+([\d\.]+)\s+([\d\.]+)\s*',line).groups())),rawtour.split('\n')))
rawmarks='''2018-01-29 00:00:00.000 13.535165989333333  48.58077572716667
29.01.2018 0:00         13.535166009        48.580775726166664
2018-01-29 00:00:01.000 13.535165977166667  48.580775749
29.01.2018 0:00         13.5351658175       48.58077575
2018-01-29 00:00:02.000 13.535165976833333  48.58077567466667
29.01.2018 0:00         13.535165988166666  48.58077563316667
2018-01-29 00:00:03.000 13.535165978333334  48.580775599
29.01.2018 0:00         13.535166127833334  48.5807756575
2018-01-29 00:00:04.000 13.535166430833334  48.5807757935
29.01.2018 0:00         13.535166510166666  48.580775819
2018-01-29 00:00:05.000 13.5351665845       48.5807758835
29.01.2018 0:00         13.5351665215       48.580775906
2018-01-29 00:00:06.000 13.535166549166666  48.58077594583333
29.01.2018 0:00         13.535166521333334  48.58077594466667
2018-01-29 00:00:07.000 13.535166487        48.580775927666664
29.01.2018 0:00         13.5351670905       48.58077611433333
2018-01-29 00:00:08.000 13.5351669075       48.5807760195
29.01.2018 0:00         13.535166444166666  48.580775919'''
marks=list(map(lambda line:list(map(lambda x:float(x),re.match(r'[^\s]+\s+[^\s]+\s+([\d\.]+)\s+([\d\.]+)\s*',line).groups())),rawmarks.split('\n')))

And [[dist(A,B,C) for A,B in zip(tour,tour[1:])] for C in marks] produces the distance matrix, which is totally empty in this case as the two coordinate sets are too far away related to the narrow perpendicular strips built on the line segments. One degree of longitude is around 111 km-s at latitude ~48 degrees. While the entire example tour is only 2.5 km long.

For speeding up the thing, https://docs.python.org/3/library/functools.html#functools.lru_cache could be used on pdist , or the c -s (length of tour-segments) could be explicitly pre-calculated and stored somewhere, the /c/c division could be postponed and temprat<0 or temprat>c**2 could be used in the if , where c**2 could be pre-calculated too. Just I lost motivation when noticed the distance issue.

---

I think you will have to check point-point distances too. See the upper part of figure: the blue point falls outside of the gray rectangles (of course they are infinite, and extend towards both directions, I just capped them for better visuals), but happens to be inside the red one, so an orthogonal-distance-only approach would say that the rightmost segment was the closest one to the point in question, while looking at point-point distances could find the green point as closest location.

However the lower part of the image is meant to illustrate that point-point distances can not be used as replacement, and it can not even really "drive" the orthogonal calculations either, as the red point being the closest corner-point to the blue one does not mean that a segment could not be closer, and the red one is not even an endpoint of that segment.

For making it work with lat-lon coordinates lots of creepy formulaes exist, I would not dare to choose one for you at the moment. https://en.wikipedia.org/wiki/Geographical_distance may be a good starting point. One shortcut you can rely on is that your trajectories contain distances in km-s, so you can calculate trip-distance of a point in a segment as a weighted sum of the trip-distances at its endpoints (instead of calculating distances directly from lat-lon coordinates).