向 python 字典添加值，只返回最后一个值

Question

Below is a list of twitter handles I am using to scrape tweets

myDict = {}

list = ['ShoePalace', 'StreetWearDealz', 'ClothesUndrCost', 'DealsPlus', 'bodega', 'FRSHSneaks', 
            'more_sneakers', 'BOOSTLINKS', 'endclothing', 'DopeKixDaily', 'RSVPGallery', 'StealSupply',
            'SneakerAlertHD', 'JustFreshKicks', 'solefed', 'SneakerMash', 'StealsBySwell', 'KicksDeals', 
            'FatKidDeals', 'sneakersteal', 'SOLELINKS', 'SneakerShouts', 'KicksUnderCost', 'snkr_twitr',
            'KicksFinder']

In the for loop below I am cycling thru each twitter handle and grabbing data. After the data is pull I am attempting to add the data to the dictionary (myDict). Currently the code is only returning a single value:

{'title': 'Ad: Nike Air Max 97 Golf 'Grass' is back in stock at Nikestore,\n\n>>': 'url'. 'example,com': 'image'. 'image,jpg': 'tweet_url'. 'example,com': 'username', 'KicksFinder': 'date'. datetime,datetime(2020, 7, 27, 11, 44, 26)}

for i in list:
    for tweet in get_tweets(i, pages=1):
        tweet_url = 'https://www.twitter.com/' + tweet['tweetUrl']
        username = tweet['username']
        date = tweet['time']
        text = tweet['text']
        title = text.split('http')[0]
        title = title.strip()
        title = title.rstrip()
        try:
            entries = tweet['entries']
            image = entries["photos"][0]
            url = entries["urls"][0]
            myDict['title'] = title
            myDict['url'] = url
            myDict['image'] = image
            myDict['tweet_url'] = tweet_url
            myDict['username'] = username
            myDict['date'] = date
        except IndexError:
            title = title
            image = ""
            link = ""
   
    return(myDict)

Answer 1

You're mutating a single dict, not adding to a list.

We can refactor your code to a handful of simpler functions that process tweepy? Tweets into dicts and others that yield processed tweet dicts for a given user.

Instead of printing the tweets at the end, you could now list.append them - or even simpler, just tweets = list(process_tweets_for_users(usernames)) :)

def process_tweet(tweet) -> dict:
    """
    Turn a Twitter-native Tweet into a dict
    """
    tweet_url = "https://www.twitter.com/" + tweet["tweetUrl"]
    username = tweet["username"]
    date = tweet["time"]
    text = tweet["text"]
    title = text.split("http")[0]
    title = title.strip()
    try:
        entries = tweet["entries"]
        image = entries["photos"][0]
        url = entries["urls"][0]
    except Exception:
        image = url = None
    return {
        "title": title,
        "url": url,
        "image": image,
        "tweet_url": tweet_url,
        "username": username,
        "date": date,
    }


def process_user_tweets(username: str):
    """
    Generate processed tweets for a given user.
    """
    for tweet in get_tweets(username, pages=1):
        try:
            yield process_tweet(tweet)
        except Exception as exc:
            # TODO: improve error handling
            print(exc)


def process_tweets_for_users(usernames):
    """
    Generate processed tweets for a number of users.
    """
    for username in usernames:
        yield from process_user_tweets(username)


usernames = [
    "ShoePalace",
    "StreetWearDealz",
    "ClothesUndrCost",
    "DealsPlus",
    "bodega",
    "FRSHSneaks",
    "more_sneakers",
    "BOOSTLINKS",
    "endclothing",
    "DopeKixDaily",
    "RSVPGallery",
    "StealSupply",
    "SneakerAlertHD",
    "JustFreshKicks",
    "solefed",
    "SneakerMash",
    "StealsBySwell",
    "KicksDeals",
    "FatKidDeals",
    "sneakersteal",
    "SOLELINKS",
    "SneakerShouts",
    "KicksUnderCost",
    "snkr_twitr",
    "KicksFinder",
]

for tweet in process_tweets_for_users(usernames):
    print(tweet)

Answer 2

It is expected you only get the results for the last value in your lists because you seem to be overwriting the results for each tweet, instead of appending them to a list. I would use defauldict(list) and then append each tweet:

from collections import defaultdict
myDict = defaultdict(list)
for i in list:
    for tweet in get_tweets(i, pages=1):
        tweet_url = 'https://www.twitter.com/' + tweet['tweetUrl']
        username = tweet['username']
        date = tweet['time']
        text = tweet['text']
        title = text.split('http')[0]
        title = title.strip()
        title = title.rstrip()
        try:
            entries = tweet['entries']
            image = entries["photos"][0]
            url = entries["urls"][0]
            myDict['title'].append(title)
            myDict['url'].append(url)
            myDict['image'].append(image)
            myDict['tweet_url'].append(tweet_url)
            myDict['username'].append(username)
            myDict['date'].append(date)
        except IndexError:
            title = title
            image = ""
            link = ""
   
    return(myDict)

Now that you have everything nice and tidy, you can put it into a nice dataframe to work with your data:

tweets_df = pd.DataFrame(tweets_df)