Converting plotly graph to dataframe in order to plot dataframe with cufflinks iplot graph

Question

The context of the problem that I am dealing with is trying to convert the results from a time series forecast, plotted with matplotlib.plotly back into a dataframe so that I can use the cufflinks library to be able to get a more interactive chart going so that I can hover over data points to get a more detailed look at the forecast.

so after training and creating a simulation the code goes:

    date_ori = pd.to_datetime(df.iloc[:, 0]).tolist()
for i in range(test_size):
    date_ori.append(date_ori[-1] + timedelta(days = 1))
date_ori = pd.Series(date_ori).dt.strftime(date_format = '%Y-%m-%d').tolist()
date_ori[-5:]


accepted_results = []
for r in results:
    if (np.array(r[-test_size:]) < np.min(df['Close'])).sum() == 0 and \
    (np.array(r[-test_size:]) > np.max(df['Close']) * 2).sum() == 0:
        accepted_results.append(r)
len(accepted_results)


accuracies = [calculate_accuracy(df['Close'].values, r[:-test_size]) for r in accepted_results]

plt.figure(figsize = (15, 5))
for no, r in enumerate(accepted_results):
    plt.plot(r, label = 'forecast %d'%(no + 1))
plt.plot(df['Close'], label = 'true trend', c = 'black')

plt.legend()
plt.title('average accuracy: %.4f'%(np.mean(accuracies)))

x_range_future = np.arange(len(results[0]))
plt.xticks(x_range_future[::30], date_ori[::30])

plt.show()

I have started to dissect the last plotting section to attempt to convert the data into a dataframe in order to plot with cufflinks as the format for cufflinks is like:

import cufflinks as cf
# data from FXCM Forex Capital Markets Ltd.
raw = pd.read_csv('http://hilpisch.com/fxcm_eur_usd_eod_data.csv',
                 index_col=0, parse_dates=True)
quotes = raw[['AskOpen', 'AskHigh', 'AskLow', 'AskClose']]
quotes = quotes.iloc[-60:]
quotes.tail()

    AskOpen AskHigh AskLow  AskClose
2017-12-25 22:00:00 1.18667 1.18791 1.18467 1.18587
2017-12-26 22:00:00 1.18587 1.19104 1.18552 1.18885
2017-12-27 22:00:00 1.18885 1.19592 1.18885 1.19426
2017-12-28 22:00:00 1.19426 1.20256 1.19369 1.20092
2017-12-31 22:00:00 1.20092 1.20144 1.19994 1.20147

qf = cf.QuantFig(
         quotes,
         title='EUR/USD Exchange Rate',
         legend='top',
         name='EUR/USD'
)

qf.iplot()

Where I have gotten so far is trying to dissect the plotly graph into a dataframe as so, these are the forecasted results:

df = accepted_results
rd = pd.DataFrame(df)
rd.T

    0   1   2   3   4   5   6   7
0   768.699985  768.699985  768.699985  768.699985  768.699985  768.699985  768.699985  768.699985
1   775.319656  775.891012  772.283885  737.763376  773.811344  785.021571  770.438252  770.464180
2   772.387081  787.562968  764.858772  737.837558  775.712162  770.660990  768.103724  770.786379
3   786.316425  779.248516  765.839603  760.195678  783.410054  789.610540  765.924561  773.466415
4   796.039144  803.113903  790.219174  770.508252  795.110376  793.371152  774.331197  786.772606
... ... ... ... ... ... ... ... ...
277 1042.788063 977.462670  1057.189696 1262.203613 1057.900621 1042.329811 1053.378352 1171.416597
278 1026.857102 975.473725  1061.585063 1307.540754 1061.490772 1049.696547 1054.122795 1117.779434
279 1029.388746 977.097765  1069.265953 1192.250498 1064.540056 1049.169295 1045.126807 1242.474584
280 1030.373147 983.650686  1070.628785 1103.139889 1053.571269 1030.669091 1047.641127 1168.965372
281 1023.118504 984.660763  1071.661590 1068.445156 1080.461617 1035.736879 1035.599867 1231.714340

then converting the x axis from

plt.xticks(x_range_future[::30], date_ori[::30])

to

df1 = pd.DataFrame((x_range_future[::30], date_ori[::30]))
df1.T
    0   1
0   0   2016-11-02
1   30  2016-12-15
2   60  2017-01-31
3   90  2017-03-15
4   120 2017-04-27
5   150 2017-06-09
6   180 2017-07-24
7   210 2017-09-05
8   240 2017-10-17
9   270 2017-11-20

lastly I have the close column and this is what I've been able to come up with for it so far

len(df['Close'].values)
252

when i use

df['Close'].values 

I get an array, I'm having problems getting this all together, the cufflinks iplot graphs are just way better, and it would be amazing if I could somehow gain the intuition to do this, I apologize in advance if I didn't try hard enough, but I'm doing my best I can't seem to find the answer no matter how many times I've searched google so I thought I would ask here.

Answer 1

This is what I did, I went through and printed indipendent strings like print(date_ori) as well as simplified it with print(len(date_ori) which in turn had all of the dates for the forecast, then i made it into a dataframe with df['date'] = pd.DataFrame(date_ori), where as with the results, I had to transpose them with df.T so they would be in a long column format rather than in a long row, so first

df = pd.DataFrame(results)
df = df.T

then

df['date'] = pd.DataFrame(date_ori)

I had trouble naming the column 0 which contained all of the predicted results so i just saved the file with

df.to_csv('yo')

then i edited the column named 0 to results and added.csv to the end, then pulled the data back into memory

then i formatted the date

format = '%Y-%m-%d'
df['Datetime'] = pd.to_datetime(df['date'], format=format)
df = df.set_index(pd.DatetimeIndex(df['Datetime']))

and dropped the un needed columns, and i guess i could add the close data that i started with to plot together now, but i got the results into the dataframe so now i can use these awesome charts. Can't believe i figured it out within 18 hours I was so lost lol.

also i dropped the experiment to just one simulation so there was only 1 row of results to deal with so i could figure it out.