Scrapy：如何用无用的值替换字典中的其他项目

Question

I'am currently scraping prices from a website, most products have maximum and minimum prices, but not all of them have minimum prices. Those who don't have minimum, trow useless values I have been replacing for no text "", but I would like to replace those empty values with maximum price (basically because if a price doesn't change minimum and maximum are the same).

The code is extensive, so I have the following libraries imported:

import os
import scrapy
from ..items import TutorialItem
import pandas as pd
from scrapy.http import Request
from scrapy.http import FormRequest
from scrapy.selector import Selector
from scrapy.utils.response import open_in_browser
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager

class KikoSpider2(scrapy.Spider):
    name = "kiko2"

    login_page = 'https://www.kikowireless.com/login'
    formdata = {'email': 'thisisan@email.com',
                 'password': 'QQntDXqK9'}

The code goes on..

The important thing comes here:

def parse_products(self, response):
        items = TutorialItem()
        category = response.meta['category']

        article_name = response.css('#content .name a::text').extract()
        article_price = [ x.replace('$', '').replace('\n', '').replace('\t', '').replace(' ', '') for x in response.css('.price::text').extract()]
        article_price_min = [x.replace('\t', '').replace(
        '$', '').replace('\n', 'n').split()[-1].replace('n', '') for x in response.css('.discount::text').extract()] 

        items['article_name'] = article_name
        items['article_price'] = article_price
        items['article_price_min'] = article_price_min
        for item in zip(article_name, article_price, article_price_min):
            scraped_info = {'supplier_url' : item[0],
                                'supplier_item_name' : item[1],
                                'max_price' : item[2],
                                'min_price' : item[3],
                                  }
                # print(scraped_info)
            df_result = pd.DataFrame.from_dict(scraped_info.items())
            print(df_result)
            yield scraped_info

the code line article_price = [ x.replace('$', '').replace('\n', '').replace('\t', '').replace(' ', '') for x in response.css('.price::text').extract()] Extracts the minimum price of the articles, what can I do to fill the blank spaces in it with the article_price corresponding to the same underlying item.

Answer 1

That's quite simple.

#this if checks if the value is not null or empty
if article_price_min:
  items['article_price_min'] = article_price_min
else:
  items['article_price_min'] = article_price