使用正则表达式提取数据并将其插入.csv文件

Question

So I am using regex to pull data from a webpage. Done.

Now I am trying to insert this data into a .csv file. No problem right?

So I am having trouble pulling my data from the loops I created to insert into the .csv file. It looks like the best way to conquer this is to create a list, and somehow insert the data into the list and write the data into the csv file. But how can I do that with my current setup?

import re
import sqlite3 as lite
import mysql.connector
import urllib.request
from bs4 import BeautifulSoup
import csv

#We're pulling info on socks from e-commerce site Aliexpress

url="https://www.aliexpress.com/premium/socks.html?SearchText=socks&ltype=wholesale&d=y&tc=ppc&blanktest=0&initiative_id=SB_20171202125044&origin=y&catId=0&isViewCP=y"

req = urllib.request.urlopen(url)
soup = BeautifulSoup(req, "html.parser")
div = soup.find_all("div", attrs={"class":"item"})

for item in div:
    title_pattern = '<img alt="(.*?)\"'
    comp = re.compile(title_pattern)
    href = re.findall(comp, str(item))
    for x in href:
        print(x)

    price_pattern = 'itemprop="price">(.*?)<'
    comp = re.compile(price_pattern)
    href = re.findall(comp, str(item))
    for x in href:
        print(x)

    seller_pattern = '<a class="store j-p4plog".*?>(.*?)<'
    comp = re.compile(seller_pattern)
    href = re.findall(comp, str(item))
    for x in href:
        print(x)

    orders_pattern = '<em title="Total Orders">.*?<'
    comp = re.compile(orders_pattern)
    href = re.findall(comp, str(item))
    for x in href:
        print(x[32:-1])

    feedback_pattern = '<a class="rate-num j-p4plog".*?>(.*)<'
    comp = re.compile(feedback_pattern)
    href = re.findall(comp, str(item))
    for x in href:
        print(x)

# Creation and insertion of CSV file

# csvfile = "aliexpress.csv"
# csv = open(csvfile, "w")
# columnTitleRow = "Title,Price,Seller,Orders,Feedback,Pair"
# csv.write(columnTitleRow)
#
# for stuff in div:
#     title = 
#     price = 
#     seller = 
#     orders = 
#     feedback = 
#     row = title + "," + price + "," + seller + "," + orders + "," + feedback + 
"," + "\n"
#     csv.write(row)

I want to be able to print these lists by their row.

Answer 1

It looks like the best way to conquer this is to create a list, and somehow insert the data into the list and write the data into the csv file. But how can I do that with my current setup?

Yes you're right. Replace your print statements with append s to a list:

data = []
for item in div:
    title_pattern = '<img alt="(.*?)\"'
    comp = re.compile(title_pattern)
    href = re.findall(comp, str(item))
    for x in href:
        data.append(x)

    price_pattern = 'itemprop="price">(.*?)<'
    comp = re.compile(price_pattern)
    href = re.findall(comp, str(item))
    for x in href:
        data.append(x)

And then later

csv.writerow(data)

From what I remember, csv.write takes a list and not a rendered CSV string anyways. That's the whole point, it takes the raw data and escapes it properly and adds the commas for you.

Edit: As explained in the comment, I misremembered the interface to csv writer. writerow takes a list, not write . Updated.