From 3D to 2D-array Basemap plot (Python)

Question

I'm trying to plot data on a map (basemap) according lon/lat arrays with the following shapes:

lon (Ntracks, Npoints)
lat (Ntracks, Npoints)
data(Ntracks, Npoints, Ncycles)

The problem is that, with the code I did below, nothing is displayed on the map (and it's terribly slow! even close to out of memory error...). I'm probably using the wrong method. Can I have your help to be able to convert this 3D-data array to a 2D one to use the pcolormesh function ?

map = Basemap(projection='mill',lat_ts=10,\
            llcrnrlon= -11.01472, \
            urcrnrlon=  2.769825, \
            llcrnrlat=  43.292, \
            urcrnrlat=  52.10833, \
            resolution='l')

# Zone
lonmin, lonmax = map.lonmin, map.lonmax
latmin, latmax = map.latmin, map.latmax       

# Missing value
misval = 99.999

# Masks on Lon/Lat arrays
lon = np.ma.masked_where(np.isnan(lon) == True, lon)
lat = np.ma.masked_where(np.isnan(lat) == True, lat)
lon = np.ma.masked_where(lon <= lonmin, lon)
lon = np.ma.masked_where(lon >= lonmax, lon)   
lat = np.ma.masked_where(lat <= latmin, lat)
lat = np.ma.masked_where(lat >= latmax, lat)

# Dimensions
Ntracks, Npoints, Ncycles = data.shape
'''
#-----------------
# TRACKS LOCATIONS 
# IT WORKS => SEE FIGURE POSTED BELOW
#-----------------
for track in range(Ntracks): 
  x, y = map( *np.meshgrid( lon[track,:], lat[track,:] ))
  y = y.T
  map.plot(x, y, marker= 'o',    \
               markersize=1.5, \
               color='#0B610B',
               markeredgecolor='#0B610B')

'''
#-----------------
# TRACKS DATA 
# => NOT WORKING !!
#-----------------
Data = np.full( (Npoints*Npoints), misval )

for track in range(Ntracks): 
  x, y = map( *np.meshgrid( lon[track,:], lat[track,:] ))
  y = y.T
  for c in range(Ncycles):
    Data[0:Npoints] = data[track,:,c]

    Data = np.ma.masked_where(np.isnan(Data) == True, Data)
    Data = np.ma.masked_where(Data >= misval, Data)

    Data = Data.reshape( (Npoints, Npoints) )

    # plot Data on the map
    map.pcolormesh(x, y, Data)


parallels = np.arange( latmin, latmax, 2., dtype= 'int')
meridians = np.arange( lonmin, lonmax, 2., dtype= 'int')

map.drawcoastlines(linewidth=1.5, color='white')
map.drawcountries(linewidth=1.5, color='grey')
map.drawparallels(parallels, labels= [1,0,0,0], color= 'black',   fontsize=16, linewidth=0)
map.drawmeridians(meridians, labels= [0,0,0,1], color= 'black', fontsize= 16, linewidth=0)
map.drawmapboundary(color='k',linewidth=2.0)
map.fillcontinents(color='#cdc5bf')

This code generates no error but no data is dispayed on the map...
Here is an example of what the code is returning in output for Ntracks=1 and Ncycles=4 :

FILE 0 TRACK 011
CYCLE 1
[[-- 60.779693603515625 60.658145904541016 ..., 0.0 0.0 0.0]
 [-- 60.779693603515625 60.658145904541016 ..., 0.0 0.0 0.0]
 [-- 60.779693603515625 60.658145904541016 ..., 0.0 0.0 0.0]
 ..., 
 [-- 60.779693603515625 60.658145904541016 ..., 0.0 0.0 0.0]
 [-- 60.779693603515625 60.658145904541016 ..., 0.0 0.0 0.0]
 [-- 60.779693603515625 60.658145904541016 ..., 0.0 0.0 0.0]]
CYCLE 2
[[-- 60.7838249206543 60.666202545166016 ..., 0.0 0.0 0.0]
 [-- 60.7838249206543 60.666202545166016 ..., 0.0 0.0 0.0]
 [-- 60.7838249206543 60.666202545166016 ..., 0.0 0.0 0.0]
 ..., 
 [-- 60.7838249206543 60.666202545166016 ..., 0.0 0.0 0.0]
 [-- 60.7838249206543 60.666202545166016 ..., 0.0 0.0 0.0]
 [-- 60.7838249206543 60.666202545166016 ..., 0.0 0.0 0.0]]
CYCLE 3
[[-- -- -- ..., 0.0 0.0 0.0]
 [-- -- -- ..., 0.0 0.0 0.0]
 [-- -- -- ..., 0.0 0.0 0.0]
 ..., 
 [-- -- -- ..., 0.0 0.0 0.0]
 [-- -- -- ..., 0.0 0.0 0.0]
 [-- -- -- ..., 0.0 0.0 0.0]]
CYCLE 4
[[-- -- -- ..., 0.0 0.0 0.0]
 [-- -- -- ..., 0.0 0.0 0.0]
 [-- -- -- ..., 0.0 0.0 0.0]
 ..., 
 [-- -- -- ..., 0.0 0.0 0.0]
 [-- -- -- ..., 0.0 0.0 0.0]
 [-- -- -- ..., 0.0 0.0 0.0]]

I should have something like this (and do a treatment close to 0° lon) Tracks locations

Answer 1

This map seems to be a very nice tool, thank you for the hint :)

If I get it right, Data should be a 2D-Array containing a value for each xy-pair? But you fill only the first Npoints entries of Data, and then you just reshape it -- so you did not write anything in any rows besides the first of your Npoints x Npoints matrix. This is probably not intended, you want to fill the i-th point into the [i, i]-element of the matrix.

So create the matrix with Data = np.full((Npoints, Npoints), np.nan) instead of the 1D-matrix with length N*N you were creating. And then np.fill_diagonal(Data, data[track, :, c])

And I'm not sure if you are right there:

If latlon keyword is set to True, x,y are intrepreted as longitude and latitude in degrees. [...]

is this not what you want?