使用Datashader绘制NumPy数组数据的最佳方法是什么？

Question

In following the Datashader example notebook demonstrating lines , the input is a Pandas DataFrame (though it seems a Dask DataFrame would work as well). My data is in a NumPy array. Can I use Datashader to plot lines from NumPy arrays without first putting them into a DataFrame?

The documentation for line glyph seems to indicate this is possible but I did not find an example. The example notebook I linked to uses Canvas.line which I did not find in the documentation.

Answer 1

I did not find a way to plot data in a NumPy array without first putting it into a DataFrame. How to do this was not especially intuitive, it seems Datashader requires the column labels to be non-numeric strings, so they can be called using the df.col_label syntax (rather than the df[col_label] syntax, perhaps there is a good reason for this though).

With the current system I had to do the following to get the NumPy array into a DataFrame with column labels Datashader would accept.

df = pd.DataFrame(data=data.T)
data_cols = ['c{}'.format(c) for c in df.columns]
df.columns = data_cols
df['x'] = x_values

y_range = data.min(), data.max()
x_range = x_values[0], x_values[-1]

canvas = datashader.Canvas(x_range=x_range, y_range=y_range, 
                           plot_height=300, plot_width=900)
aggs = collections.OrderedDict((c, canvas.line(df, 'q', c)) for c in data_cols)

merged = xarray.concat(saxs_aggs.values(), dim=pd.Index(cols, name='cols'))
saxs_img = datashader.transfer_functions.shade(merged.sum(dim='cols'), 
                                               how='eq_hist')

Note that the data_cols variable was important to use, rather than simply df.columns , because it had to exclude the x column (not initially intuitive).

Here is an example of the resulting with axes added using bokeh.

Answer 2

The OrderedDict and xarray.concat method was incredibly slow when applied to many data curves. The following example demonstrates a much faster method. See this GitHub issue for timings and further discussion.

import pandas as pd
import numpy as np
import datashader
import bokeh.plotting
import collections
import xarray
import time
from bokeh.palettes import Colorblind7 as palette

bokeh.plotting.output_notebook()

# create some data worth plotting
nx = 50
x = np.linspace(0, np.pi * 2, nx)
y = np.sin(x)
n = 10000
data = np.empty([n+1, len(y)])
data[0] = x
prng = np.random.RandomState(123)

# scale the data using a random normal distribution
offset = prng.normal(0, 0.1, n).reshape(n, -1)
data[1:] = y
data[1:] += offset

# make some data noisy
n_noisy = prng.randint(0, n,5)
for i in n_noisy:
    data[i+1] += prng.normal(0, 0.5, nx)

dfs = []
split = pd.DataFrame({'x': [np.nan]})
for i in range(len(data)-1):
    x = data[0]
    y = data[i+1]
    df = pd.DataFrame({'x': x, 'y': y})
    dfs.append(df)
    dfs.append(split)

df = pd.concat(dfs, ignore_index=True)   

canvas = datashader.Canvas(x_range=x_range, y_range=y_range, 
                           plot_height=300, plot_width=300)
agg = canvas.line(df, 'x', 'y', datashader.count())
img = datashader.transfer_functions.shade(agg, how='eq_hist')
img