How to add Items to a Single Dictionary from Multiple For Loops?

Question

I started Learning Python Recently. What I am basically doing is Scraping data from Website and adding to a list of dictionaries, This is what the final structure should look like:

This is basically my scraping code. I had to use two for loops since, the element to target are present at different positions on the webpage( One for Title and Another for Description )

jobslist=[]
    for item in title:
        MainTitle = item.text
        mydict = {
        'title' : MainTitle,
        }
    jobslist.append(mydict)

for i in link:
    links = i['href']
    r2 = requests.get(links, headers = headers)
    soup2 = BeautifulSoup(r2.content,'lxml')
    entry_content = soup2.find('div', class_ ='entry-content')
    mydict= {
    'description' : entry_content
    }
jobslist.append(mydict) 

Finally Saving to a CSV (pandas library used where pd is the import)

df = pd.DataFrame(jobslist)
df.to_csv('data.csv') 

But, the Output is quite strange. The description are added below the Titles and not side by side. This is the Screenshot:

How can I align it side by side?

Answer 1

Disclaimer: It's hard to give a perfect answer because your code is not reproducible; I have no idea what your date looks like, nor what you're trying to do, so I can't really test anything.

From what I understand of your code, it looks like the dictionaries are completely unnecessary. You have a list of titles, and a list of descriptions. So be it:

titles_list = []
    for item in title:
        titles_list.append(item.text)

descriptions_list = []
for i in link:
    links = i['href']
    r2 = requests.get(links, headers = headers)
    soup2 = BeautifulSoup(r2.content,'lxml')
    entry_content = soup2.find('div', class_ ='entry-content')
    descriptions_list.append(entry_content)

df = pd.DataFrame(data = {'title': titles_list, 'description': descriptions_list}) # here we use a dict of lists instead of a list of dicts
df.to_csv('data.csv')