How to create a new data frame with each data point as a new column?

Question

I am trying to scrape tables from a web page. The web page contains links to data in tables. Basically, I am writing a for loop to get the table from each link and concatenating with the other table. To do this I need to transpose each data point as a column.

df= 2004 | 2006 | 2007 | 2008 ------------------------ GrowthRate 5% | 7% |-5% | 5% Earnings 234 | 449 | -300 | 448 EPS 17.6 |11.8 | 16.8 | 500 P\\E 14.08|12.04 |11.37 | 348

I want to make it as

df = GrowthRate_2004 GrowthRate_2006 GrowthRate_2007 GrowthRate_2008 Earnings_2004 Earnings_2006 Earnings_2007 Earnings_2008 EPS_2004 EPS_2006 EPS_2007 EPS_2008 P/E_2004 P/E_2006 P/E_2007 P/E_2008 5% 7% -5% 5% 234 449 -300 448 17.6 11.8 16.8 500 14.08 12.04 11.37 348

is there any easy way to do this?

Answer 1

A sketch of the answer (for python) would be:

• read the data into a pandas.DataFrame
• with df.unstack() you get the form you are looking for

Answer 2

Maybe not so elegant solution, but works:

df_ = pd.DataFrame(data = np.broadcast_to(df.columns.values.reshape(-1, 1).astype(str), df.shape), index = df.columns.values, columns = df.index.values)
df_ = pd.get_dummies(df_)
df_ = pd.DataFrame(data = df.values.flatten().reshape(1, -1), columns=df_.columns)
df_ = pd.DataFrame(data = np.broadcast_to(df.index.values.reshape(-1, 1), df.shape), columns = df.columns)

where df is the dataframe you're converting from.