简体   繁体   中英

Webscrape using Python and BeautifulSoup - Error saving to csv file

I am attempting to write a script that will scrape the Name, Role, and Phone numbers of real estate agents from this website .

My code:

containers = page_soup.findAll("div",{"class":"card horizontal-split vcard"})

filename = "agents.csv"
f = open(filename, "w")

headers = "name, role, number\n" 

f.write(headers)

for container in containers:
    agent_name = container.findAll("li", {"class":"agent-name"})
    if agent_name:
        name = agent_name[0].text

    agent_role = container.findAll("li", {"class":"agent-role"})
    if agent_role:
        role = agent_role[0].text

    filterfn = lambda x: 'href' in x.attrs and x['href'].startswith("tel")
    phones = list(map(lambda x: x.text,filter(filterfn,container.findAll("a"))))

    print("name: " + name)
    print("role: " + role)
    print("phones:" + repr(phones))

    f.write(name + "," +role + "," + phones.replace(",", "|") + "," + "\n")

f.close()

My code worked within the terminal before attempting to save it to a csv file that I can open in excel. However, now I'm receiving the two error messages:

TypeError: must be str, not list
f.write(name + "," +role + "," + phones.replace(",", "|") + "," + "\n")

and

f.write(name + "," +role + "," + phones.replace(",", "|") + "," + "\n")
AttributeError: 'list' object has no attribute 'replace'

**Note, I am replacing "," with "|" to avoid creating extra columns inside the csv file.*

As the error mentions, phones is a list that doesn't have replace() method. You can use .join() instead to join the elements of the list with the specified separator (in this case | ):

f.write(name + "," +role + "," + '|'.join(phones) + "," + "\n")

for example:

>>> phones = ['123', '321', '123']
>>> '|'.join(phones)
'123|321|123'

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM