Numpy 形状不匹配

Question

Good day,

Sorry for the long detail.., I have 258 input files which includes 15 years of data. The purpose of this script is extract a particular column assigned as a variable within a user entered start and end year, The script was originally structured to extract the whole column in that time period, but now I am further filtering the desired values to be extracted as I am only interested in values between 0-1 in that column. But, since the original script was designed to handle 731 samples (365 days + 365 days + 1). the script automatically looks for 731 sample values, And since I am not experienced with the Netcdf function in Scipy and Numpy modules that much. I don't know how to restructure this part of the script according to the filtered days having the desired values, As each year will have different number of values between 0-1, I am having trouble assigning the initial sample size according to the filtered days found by the script, I did found a way around it by giving dummy values of -1 to avoid broadcasting issues and to meet the requirement of 731 sample size. but when I open the Netcdf file in ArcGIS I see this. which is okay as the max and min values are 1 and -1 respectively? so I am not able to see my desired values in the raster properly as ArcGIS only shows most of the -1s values in the display, I hope I am able to explain my problem, Can anyone please help me with structuring the sample size according to the filtered values found by the script. Although, I did find NetCDF4Excel. an Excel add-in to just delete the dummy values stored in the,nc file in Excel. but it seems like this add-in doesn't work on 64-bit systems, I know the problem lies with days. but I don't know how to fix it. Oh just forgot to mention that it's a 3D array containing the X coordinate, Y coordinate and date/time. enter image description here

# for what date?
start_year = input("Enter start year:")
end_year = input("End year:")

inidate = datetime.date(start_year,1,1)
enddate = datetime.date(end_year,12,31)

days = enddate.toordinal() - inidate.toordinal()+1 

This is the part of the script I use for filtering values in a particular column:

for l in lixo:
        if int(l.split("\t")[0]) in range(inidate.year, enddate.year+1):
            if var==3:
                if previousValue==-10:                    
                    previousValue=float(l.split("\t")[var])
                    dado.append(-1)
                else:
                    currentValue=float(l.split("\t")[var])
                    if currentValue==0 and previousValue>0:
                        dado[-1]=previousValue
                        dado.append(currentValue)
                    else:
                        dado.append(-1)
                    previousValue=currentValue
            else:
                dado.append(float(l.split("\t")[var]))
        # putting data inside array.
        # Since data has lat & lon fixed uses dimension [:,lat_index,lon_index]

    print dado

The whole script is as follows:

import os
import sys
# handle dates...
import datetime
# SciPy netCDF and NumPy
from scipy.io.netcdf import *
from numpy import *

skip_lines = 6

# building file list and sorted lat lon list
file_list = os.listdir(sys.argv[1])

lat_t = []
lon_t = []
lat = []
lon = []

for f in file_list:
    lat_t.append(float(f.split("_")[1]))
    lon_t.append(float(f.split("_")[2]))

for i in lat_t:
    if i not in lat:
        lat.append(i)

for i in lon_t:
    if i not in lon:
        lon.append(i)
# putting in order. Lat should be from top to bottom
# lon from left to right 
lon.sort()
lat.sort()
lat.reverse()

del(lat_t)
del(lon_t)

#determining the parameter to use
print "Choose output parameter"
varini = input('Choose output (1 a 8)>')

#getting the column right
if int (varini) < 8:
    var = varini + 2
#set name of out_file. Named after parameter choice
if var == 3:
    var_txt = "ABCD"
    var_name = "ABCD"
# for what date?
start_year = input("Enter start year:")
end_year = input("End year:")

inidate = datetime.date(start_year,1,1)
enddate = datetime.date(end_year,12,31)

days = enddate.toordinal() - inidate.toordinal()+1

print "Go grab a coffee, this could take a while..."

#
# create array containig all data
# This is going to be huge. Create an array with -9999 (NoData)
# Then populate the array by reading each input file
#

all_data = zeros([days,len(lat),len(lon)], float)-9999

c = len(file_list)

# for each file in list
for f in file_list:
    # get lat & lon and it's index
    latitude = float(f.split("_")[1])
    longitude = float(f.split("_")[2])
    lat_id = lat.index(latitude)
    lon_id = lon.index(longitude)

    print "%i files to write." % c
    c = c -1

    infile = open(sys.argv[1]+f, "r")
    # here we skip the number of header lines
    # variable set at the beginning of the code
    lixo = infile.readlines()[skip_lines:]
    infile.close()
    dado = []
    previousValue = -10

    for l in lixo:
        if int(l.split("\t")[0]) in range(inidate.year, enddate.year+1):
            if var==3:
                if previousValue==-10:                    
                    previousValue=float(l.split("\t")[var])
                    dado.append(-1)
                else:
                    currentValue=float(l.split("\t")[var])
                    if currentValue==0 and previousValue>0:
                        dado[-1]=previousValue
                        dado.append(currentValue)
                    else:
                        dado.append(-1)
                    previousValue=currentValue
            else:
                dado.append(float(l.split("\t")[var]))

        # putting data inside array.
        # Since data has lat & lon fixed uses dimension [:,lat_index,lon_index]

    print dado
all_data[:,lat_id,lon_id] = dado

# writing NetCDF
#

ncfile = netcdf_file(var_txt+".nc", "w")

ncfile.Conventions = "COARDS"
ncfile.history = "Created using flux2cdf.py. " + datetime.date.today().isoformat()
ncfile.production = "ABCD output"

ncfile.start_date = inidate.isoformat()
ncfile.end_date = enddate.isoformat()

#create dimensions
ncfile.createDimension("X", len(lon))
ncfile.createDimension("Y", len(lat))
ncfile.createDimension("T", days)

#create variables
latvar = ncfile.createVariable("Y", "f4", ("Y",))
latvar.long_name = "Latitude"
latvar.units = "degrees_north"
latvar[:] = lat

lonvar = ncfile.createVariable("X", "f4", ("X",))
lonvar.long_name = "Longitude"
lonvar.units = "degrees_east"
lonvar[:] = lon

timevar = ncfile.createVariable("T", "f4", ("T",))
timevar.long_name = "Time"
timevar.units = "days since " + inidate.isoformat()
timevar[:] = range(0, days)

data_var = ncfile.createVariable(var_txt, "f4", ("T","Y","X"))
data_var.long_name = var_name+" calculated by ABCD"
data_var.missing_value = -9999.0
data_var.units = "milimeters"
data_var[:] = all_data

ncfile.close()

Sample data that's written in the.nc file with the required filtered values have been highlighted in yellow shown below:

enter image description here

Sample data:

245 files to write.
[-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0.0978, 0.0, -1, -1, -1, -1, -1, -1, -1, -1, 0.3112, 0.0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0.3112, 0.0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0.0978, 0.0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0.1334, 0.0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 0.0978, 0.0, -1, -1, -1, -1, -1, 0.0978, 0.0, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]
244 files to write.

Answer 1

It turns out there is a much easier way to extract particular days of interest containing those days respective values by simply using this time slice tool in ArcGIS https://support.esri.com/en/technical-article/000011318 . Now, I am able to extract individual raster files of each day of interest.