如何使用 plotly 为 dask 数据框绘制交互式图

Question

I have a Dask data frame that have 30 partitions, (each partition) have 100 million raws of data. total number of raws in the whole dask dataframe is 400 million raws. I would like to plot all of the Daskframe in one plot using plotly. How would I go about achieving this? The end result is an over all plot of the data and if I want to explor a specific region of the data I can zoom in and pan. My data size is about 3.5 GB, and they wont fot in memory.

Answer 1

3.5GB is a fairly small size, so should fit in memory. If it doesn't, there are a few strategies:

1. reduce data by selecting only columns of interest, load into memory and plot data with standard libraries:

from dask.dataframe import read_parquet

df = read_parquet(path_to_file, columns=specific_cols).compute()

2. datashader - https://plotly.com/python/datashader/