Create and assign values to a variable in a for loop with the same name of the list element

Question

this is my first question in this page.

I have a list that helps me web-scraping through the elements of the list:

list = ["web-page","web-page1", "web-page2"]

Then there is the Web-Scrape Procces that iterates over the list of webpages to scrape. I have code inside this procces, a little procces to clean the scrape for each web-page and concatenate this results for a final dataframe.

result = pd.concat([characteristic1, characteristic2], axis=1, sort=False)
result2 = pd.concat([result, characteristic3], axis=1, sort=False)

Now I want to create a final DataFrame that with the same name of the web-page it has scraped (eg "web-page") but I just don't know how to do it.

df = pd.concat([result2, characteristic4], axis=1, sort=False)

This would give me only the results of the last web-page I've scraped because the for loop iterates and erases the "df" variable. How can I make a final dataframe for each web? Thanks in advance!

Answer 1

You can create a dictionary to store your result.

result = {}

# In the loop   
result[pagename] = df

Or you can put them in a list

result = []

# In the loop
result.append(df)