如何防止此方法返回一长串的None值？

Question

This script determines which websites out of a possible number of websites has the data that I want. z runs all of the potential websites through data_grabber(item) , which returns a list of the indexes of all the valid URLs. It also includes None values for the websites that weren't valid.

import requests

a = range(0, 10)
b = 'http://www.grandexchangecentral.com/include/gecgraphjson.php?jsid=%r'

websites = []
for i in a:
    result = b % a[i]
    websites.append(result)

def data_grabber(item): 
    url = item
    r = requests.get(url, headers={'Referer': 'www.grandexchangecentral.com'})
    data = r.json
    if data != None:
        return websites.index(item)

z = [data_grabber(x) for x in websites]
z1 =  filter(None, z) 

This returns [None, None, 2, None, None, None, 6, None, 8, None] , which z1 shortens to [2, 6, 8] . The only problem with this is that the list of possible websites can get very long, and so it can take a few minutes to generate many None placeholders in z . Would it be possible to make prevent z from generating None items? Thanks in advance for any help.

EDIT: Now I realize that it's actually data_grabber taking most of the time. Still nice to know this though.

Answer 1

I agree with @DSM that it seems unlikely this is costing you too much time. But this will avoid that:

filter(None, (data_grabber(x) for x in websites))

Using the parentheses instead of brackets will make a generator of data_grabber results, which filter will then consume, building up the result list without ever making the intermediate list with None s in it.

---

Just a test of your claim about timings:

>>> %timeit filter(None, [None for x in range(100000)])
100 loops, best of 3: 9.22 ms per loop

Not exactly the same thing, but this makes a list of 100,000 None s and then filters them all out in 9 milliseconds on my computer.