条件语句语法

Question

I'm writing a web crawler at the moment and my Python is rusty as hell, so I'm simply wondering whether or not there is a shorter syntax to accomplish the following...

def parse(self, response):
    prc_path = '//span[@class="result-meta"]/span[@class="result-price"]/text()'
    sqf_path = '//span[@class="result-meta"]/span[@class="housing"]/text()'
    loc_path = '//span[@class="result-meta"]/span[@class="result-hood"]/text()'
    prc_resp = response.xpath(prc_path).extract_first()
    sqf_resp = response.xpath(sqf_path).extract_first()
    loc_resp = response.xpath(loc_path).extract_first()
    if sqf_resp and loc_resp:
        yield {
            'prc': response.xpath(prc_path).extract_first(),
            'sqf': response.xpath(sqf_path).extract_first(),
            'loc': response.xpath(loc_path).extract_first()
        }
    elif sqf_resp:
        yield {
            'prc': response.xpath(prc_path).extract_first(),
            'sqf': response.xpath(sqf_path).extract_first()
        }
    else:
        yield {
            'prc': response.xpath(prc_path).extract_first(),
            'loc': response.xpath(loc_path).extract_first()
        }

As you can see, there is quite a bit of repetition and I'd like to remain as DRY as possible.

Answer 1

You can create the dictionary and then add the appropriate entries to it.

result = { 'prc': response.xpath(prc_path).extract_first() }
if sqf_path:
    result['sqf'] = response.xpath(sqf_path).extract_first()
if loc_path:
    result['loc'] = response.xpath(loc_path).extract_first()
yield result

You could also factor out the extract_path bit with a dict comprehension.

result = { 'prc': prc_path, 'sqf': sqf_path, 'loc': loc_path }
yield { key : response.xpath(value).extract_first()
          for (key, value) in result.items() if value }

In earlier versions of Python, this would be:

result = { 'prc': prc_path, 'sqf': sqf_path, 'loc': loc_path }
yield dict((key, response.xpath(value).extract_first())
          for (key, value) in result.items() if value)

Answer 2

I'd go with a lookup map:

def parse(self, response):
    # initialize your prc_path/sqf_path/loc_path here
    lookup_map = {"prc": prc_path, "sqf": sqf_path, "loc": loc_path}  # add as many as needed
    return {k: response.xpath(v).extract_first() for k, v in lookup_map.items() if v}