如何将包含不同位置的时间序列数据的多个 Pandas 数据帧合并到一个 X 数组中？

Question

I started with a downloaded.csv file that had the columns 'Series Name' then 'Country Name' (with the country name rows being repeated for each different series) then annual timesteps of the data in the following columns. I have split this into a series of Pandas dataframes, each with an index of the Country Name then the data for a single Series Name(variable) by timestep, these are saved as Pickle files by their Series Name.

For example:

¦ Country name ¦ 1960 ¦ 1961 ¦ 1962 ¦ ... ¦ 2021 ¦
___
¦ Albania      ¦ 1000 ¦ 1001 ¦ 1002 ¦ ... ¦ 1061 ¦

¦ Andorra      ¦ 2000 ¦ 2001 ¦ 2001 ¦ ... ¦ 2061 ¦

etc

I would like to build a X-array from these using 'Country Name' as one coordinate and 'Year' as another (time) coordinate then add the data from each series as a different variable.

The Country Name index column and the 'Year' column headers are the same in each dataframe, however, there are a different number of instances of Country Name compared to Year.

I am stuck on what I think should be the first step, of putting a single Series Pandas Dataframe into X-array.

I have tried the code below, both using the 'times' coordinate just generated from a list and from the current row-headers in the dataframe. However, the code really wants the two coordinates to both be the same length, which they aren't as they aren't x,y for a single point but counttry1, time1 has a value, country2 time2 has a value, country1, time2 has a value and so on.

year = list(range(1960,2020))

data = series1

locs = series1['Country Name']

times = year

array = xr.DataArray(data, coords=[times, locs], dims=["time", "space"])

This gives the error:

ValueError: conflicting sizes for dimension 'time': length 217 on the data but length 60 on coordinate 'time'

The ultimate aim of this is to combine/compare this data with spatial time series data from a C.net file that I have imported, hence wanting to use X-array.

Answer 1

I think the issue is that data in your example above is a one dimensional array, since it comes from a Series object. Based on the DataFrame example you provided, I think something like the following would achieve what you want (perhaps modulo some data cleaning to convert the string year labels to integers, etc.):

melted = df.melt(
    id_vars=["Country Name"],
    value_vars=[f"{year} [YR{year}]" for year in range(1960, 1965)],
    var_name="year",
    value_name="population"
)
ds = melted.set_index(["Country Name", "year"]).to_xarray()

• df.melt stacks the individual population columns for each year into a single column, creates a new year column to represent the year coordinate, and expands the country column to also appropriately label each population value in the stacked column.
• melted.set_index then creates a MultiIndex out of the "Country Name" and "year" columns. This allows pandas and xarray to recognize that this DataFrame is really a Dataset with a 2D data variable when to_xarray is called.

Answer 2

As Spencer Clark writes, the function "df.melt" will help you with exactly this task.

I think you should accept his answer, or clarify where further issues arise