堆栈内存溢出
  简体   繁体   English

使用 scholarly.pprint 时出现 StopIteration 错误 function

[英]StopIteration Error while using scholarly.pprint function

 原文  2021-04-17 12:20:02  2   1    python/ web-scraping/ web-crawler/ google-scholar

问题描述

I am trying to extract Google Scholar public profiles of certain professors.我正在尝试提取某些教授的谷歌学术公开资料。

I have a list of professors' names and I am using it with help of a scholarly package for scraping their public profile information.我有一份教授姓名列表，我在scholarly package 的帮助下使用它来抓取他们的公开资料信息。 However, I am stuck with an error.但是，我遇到了一个错误。 I am only able to retrieve information for the first name in the professor_list and not the subsequent ones.我只能检索professor_list中第一个名字的信息，而不能检索后面的名字。

for name in professor_list:
    search_query = scholarly.search_author(name)
    scholarly.pprint(next(search_query))

Output: Output：

{'affiliation': 'Deakin University',
 'citedby': 2528,
 'email_domain': '@deakin.edu.au',
 'filled': False,
 'interests': ['Lynn Batten'],
 'name': 'Lynn Batten',
 'scholar_id': 'Tmg0T9sAAAAJ',
 'source': 'SEARCH_AUTHOR_SNIPPETS',
 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=Tmg0T9sAAAAJ'}
---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
<ipython-input-242-5b96571c0972> in <module>
      1 for name in professor_list:
      2     search_query = scholarly.search_author(name)
----> 3     scholarly.pprint(next(search_query))

StopIteration:

1 个解决方案

解决方案1
 0  2022-03-13 09:21:57

Although, scholarly.pprint(next(search_query)) should be working, you can add default value None for next() method in case nothing is found, eg next(search_query, None) :虽然scholarly.pprint(next(search_query))应该可以工作，但您可以为next()方法添加默认值None以防万一找不到任何内容，例如next(search_query, None) ：

from scholarly import scholarly

professor_list = ["Marty Banks, Berkeley",
                  "Adam Lobel, Blizzard",
                  "Daniel Blizzard, Blizzard",
                  "Shuo Chen, Blizzard",
                  "Ian Livingston, Blizzard",
                  "Minli Xu, Blizzard"]

for professor_name in professor_list:
    search_query = scholarly.search_author(name=professor_name)
    scholarly.pprint(next(search_query, None))

More information about StopIteration by Martijn Pieters .有关Martijn Pieters 的StopIteration的更多信息。

Full output:完整 output：

{'affiliation': 'Professor of Vision Science, UC Berkeley',
 'citedby': 22559,
 'email_domain': '@berkeley.edu',
 'filled': False,
 'interests': ['vision science', 'psychology', 'human factors', 'neuroscience'],
 'name': 'Martin Banks',
 'scholar_id': 'Smr99uEAAAAJ',
 'source': 'SEARCH_AUTHOR_SNIPPETS',
 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=Smr99uEAAAAJ'}
{'affiliation': 'Blizzard Entertainment',
 'citedby': 3050,
 'email_domain': '@AdamLobel.com',
 'filled': False,
 'interests': ['Gaming', 'Emotion regulation'],
 'name': 'Adam Lobel',
 'scholar_id': '_xwYD2sAAAAJ',
 'source': 'SEARCH_AUTHOR_SNIPPETS',
 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=_xwYD2sAAAAJ'}
{'affiliation': '',
 'citedby': 873,
 'email_domain': '',
 'filled': False,
 'interests': ['Daniel Blizzard'],
 'name': 'Daniel Blizzard',
 'scholar_id': 'dk4LWEgAAAAJ',
 'source': 'SEARCH_AUTHOR_SNIPPETS',
 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=dk4LWEgAAAAJ'}
{'affiliation': 'Senior Data Scientist, Blizzard Entertainment',
 'citedby': 656,
 'email_domain': '@cs.cornell.edu',
 'filled': False,
 'interests': ['Machine Learning', 'Data Mining', 'Artificial Intelligence'],
 'name': 'Shuo Chen',
 'scholar_id': 'OBf4YnkAAAAJ',
 'source': 'SEARCH_AUTHOR_SNIPPETS',
 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=OBf4YnkAAAAJ'}
{'affiliation': 'Blizzard Entertainment',
 'citedby': 620,
 'email_domain': '@usask.ca',
 'filled': False,
 'interests': ['Human-computer interaction',
               'User Experience',
               'Player Experience',
               'User Research',
               'Games'],
 'name': 'Ian Livingston',
 'scholar_id': 'xBHVqNIAAAAJ',
 'source': 'SEARCH_AUTHOR_SNIPPETS',
 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=xBHVqNIAAAAJ'}
{'affiliation': 'Blizzard Entertainment',
 'citedby': 502,
 'email_domain': '@blizzard.com',
 'filled': False,
 'interests': ['Game', 'Machine Learning', 'Data Science', 'Bioinformatics'],
 'name': 'Minli Xu',
 'scholar_id': 'QST5iogAAAAJ',
 'source': 'SEARCH_AUTHOR_SNIPPETS',
 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=QST5iogAAAAJ'}

Alternatively, you can iterate one more time over scholarly.search_author() results to make it work:或者，您可以在scholarly.search_author()结果上再迭代一次以使其工作：

from scholarly import scholarly
import json

professor_list = ["Marty Banks, Berkeley",
                  "Adam Lobel, Blizzard",
                  "Daniel Blizzard, Blizzard",
                  "Shuo Chen, Blizzard",
                  "Ian Livingston, Blizzard",
                  "Minli Xu, Blizzard"]

professor_results = []

for professor_name in professor_list:
    for professor_result in scholarly.search_author(name=professor_name):
        professor_results.append({
            "name": professor_result.get("name"),
            "affiliations": professor_result.get("affiliation"),
            "email_domain": professor_result.get("email_domain"),
            "interests": professor_result.get("interests"),
            "citedby": professor_result.get("citedby")
        })

print(json.dumps(professor_results, indent=2, ensure_ascii=False))

Full output:完整 output：

[
  {
    "name": "Martin Banks",
    "affiliations": "Professor of Vision Science, UC Berkeley",
    "email_domain": "@berkeley.edu",
    "interests": [
      "vision science",
      "psychology",
      "human factors",
      "neuroscience"
    ],
    "citedby": 22559
  },
  {
    "name": "Adam Lobel",
    "affiliations": "Blizzard Entertainment",
    "email_domain": "@AdamLobel.com",
    "interests": [
      "Gaming",
      "Emotion regulation"
    ],
    "citedby": 3050
  },
  {
    "name": "Daniel Blizzard",
    "affiliations": "",
    "email_domain": "",
    "interests": [
      "Daniel Blizzard"
    ],
    "citedby": 873
  },
  {
    "name": "Shuo Chen",
    "affiliations": "Senior Data Scientist, Blizzard Entertainment",
    "email_domain": "@cs.cornell.edu",
    "interests": [
      "Machine Learning",
      "Data Mining",
      "Artificial Intelligence"
    ],
    "citedby": 656
  },
  {
    "name": "Ian Livingston",
    "affiliations": "Blizzard Entertainment",
    "email_domain": "@usask.ca",
    "interests": [
      "Human-computer interaction",
      "User Experience",
      "Player Experience",
      "User Research",
      "Games"
    ],
    "citedby": 620
  },
  {
    "name": "Minli Xu",
    "affiliations": "Blizzard Entertainment",
    "email_domain": "@blizzard.com",
    "interests": [
      "Game",
      "Machine Learning",
      "Data Science",
      "Bioinformatics"
    ],
    "citedby": 502
  }
]

Another alternative is to use Google Scholar Profiles API from SerpApi.另一种选择是使用来自 SerpApi 的Google Scholar Profiles API。 It's a paid API with a free plan that handles scaling, bypasses blocks from search engines via dedicated proxies and CAPTCHA solving services.这是一个付费的 API 和一个免费的计划，可以处理缩放，通过专用代理和 CAPTCHA 解决服务绕过搜索引擎的块。 Check out the playground .看看操场。

Example code to integrate:要集成的示例代码：

from serpapi import GoogleScholarSearch
import json

professor_list = ["Marty Banks, Berkeley",
                  "Adam Lobel, Blizzard",
                  "Daniel Blizzard, Blizzard",
                  "Shuo Chen, Blizzard",
                  "Ian Livingston, Blizzard",
                  "Minli Xu, Blizzard"]

for professor_name in professor_list:
    params = {
        "api_key": "Your SerpApi API key",
        "engine": "google_scholar_profiles",
        "hl": "en",
        "mauthors": professor_name
    }

    search = GoogleScholarSearch(params)
    results = search.get_dict()

    for result in results["profiles"]:
        print(json.dumps(result, indent=2))

Full output:完整 output：

{
  "name": "Martin Banks",
  "link": "https://scholar.google.com/citations?hl=en&user=Smr99uEAAAAJ",
  "serpapi_link": "https://serpapi.com/search.json?author_id=Smr99uEAAAAJ&engine=google_scholar_author&hl=en",
  "author_id": "Smr99uEAAAAJ",
  "affiliations": "Professor of Vision Science, UC Berkeley",
  "email": "Verified email at berkeley.edu",
  "cited_by": 22559,
  "interests": [
    {
      "title": "vision science",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Avision_science",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:vision_science"
    },
    {
      "title": "psychology",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Apsychology",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:psychology"
    },
    {
      "title": "human factors",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Ahuman_factors",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:human_factors"
    },
    {
      "title": "neuroscience",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aneuroscience",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:neuroscience"
    }
  ],
  "thumbnail": "https://scholar.google.com/citations/images/avatar_scholar_56.png"
}
{
  "name": "Adam Lobel",
  "link": "https://scholar.google.com/citations?hl=en&user=_xwYD2sAAAAJ",
  "serpapi_link": "https://serpapi.com/search.json?author_id=_xwYD2sAAAAJ&engine=google_scholar_author&hl=en",
  "author_id": "_xwYD2sAAAAJ",
  "affiliations": "Blizzard Entertainment",
  "email": "Verified email at AdamLobel.com",
  "cited_by": 3050,
  "interests": [
    {
      "title": "Gaming",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Agaming",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:gaming"
    },
    {
      "title": "Emotion regulation",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aemotion_regulation",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:emotion_regulation"
    }
  ],
  "thumbnail": "https://scholar.googleusercontent.com/citations?view_op=small_photo&user=_xwYD2sAAAAJ&citpid=3"
}
https://serpapi.com/search
{
  "name": "Daniel Blizzard",
  "link": "https://scholar.google.com/citations?hl=en&user=dk4LWEgAAAAJ",
  "serpapi_link": "https://serpapi.com/search.json?author_id=dk4LWEgAAAAJ&engine=google_scholar_author&hl=en",
  "author_id": "dk4LWEgAAAAJ",
  "affiliations": "",
  "cited_by": 873,
  "thumbnail": "https://scholar.google.com/citations/images/avatar_scholar_56.png"
}
{
  "name": "Shuo Chen",
  "link": "https://scholar.google.com/citations?hl=en&user=OBf4YnkAAAAJ",
  "serpapi_link": "https://serpapi.com/search.json?author_id=OBf4YnkAAAAJ&engine=google_scholar_author&hl=en",
  "author_id": "OBf4YnkAAAAJ",
  "affiliations": "Senior Data Scientist, Blizzard Entertainment",
  "email": "Verified email at cs.cornell.edu",
  "cited_by": 656,
  "interests": [
    {
      "title": "Machine Learning",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Amachine_learning",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:machine_learning"
    },
    {
      "title": "Data Mining",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Adata_mining",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:data_mining"
    },
    {
      "title": "Artificial Intelligence",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aartificial_intelligence",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:artificial_intelligence"
    }
  ],
  "thumbnail": "https://scholar.googleusercontent.com/citations?view_op=small_photo&user=OBf4YnkAAAAJ&citpid=1"
}
{
  "name": "Ian Livingston",
  "link": "https://scholar.google.com/citations?hl=en&user=xBHVqNIAAAAJ",
  "serpapi_link": "https://serpapi.com/search.json?author_id=xBHVqNIAAAAJ&engine=google_scholar_author&hl=en",
  "author_id": "xBHVqNIAAAAJ",
  "affiliations": "Blizzard Entertainment",
  "email": "Verified email at usask.ca",
  "cited_by": 620,
  "interests": [
    {
      "title": "Human-computer interaction",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Ahuman_computer_interaction",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:human_computer_interaction"
    },
    {
      "title": "User Experience",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Auser_experience",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:user_experience"
    },
    {
      "title": "Player Experience",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aplayer_experience",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:player_experience"
    },
    {
      "title": "User Research",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Auser_research",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:user_research"
    },
    {
      "title": "Games",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Agames",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:games"
    }
  ],
  "thumbnail": "https://scholar.google.com/citations/images/avatar_scholar_56.png"
}
{
  "name": "Minli Xu",
  "link": "https://scholar.google.com/citations?hl=en&user=QST5iogAAAAJ",
  "serpapi_link": "https://serpapi.com/search.json?author_id=QST5iogAAAAJ&engine=google_scholar_author&hl=en",
  "author_id": "QST5iogAAAAJ",
  "affiliations": "Blizzard Entertainment",
  "email": "Verified email at blizzard.com",
  "cited_by": 502,
  "interests": [
    {
      "title": "Game",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Agame",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:game"
    },
    {
      "title": "Machine Learning",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Amachine_learning",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:machine_learning"
    },
    {
      "title": "Data Science",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Adata_science",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:data_science"
    },
    {
      "title": "Bioinformatics",
      "serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Abioinformatics",
      "link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:bioinformatics"
    }
  ],
  "thumbnail": "https://scholar.googleusercontent.com/citations?view_op=small_photo&user=QST5iogAAAAJ&citpid=14"
}

Disclaimer, I work for SerpApi.免责声明，我为 SerpApi 工作。

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 使用Python中的学术模块在搜索查询期间进行StopIteration - StopIteration during search query using scholarly module in Python 使用多个 if 条件和 next() 时的 StopIteration 错误/异常/错误 - StopIteration error/ exception/ bug while using multiple if condition and next() 在生成器函数中获取 StopIteration 错误 - Getting StopIteration error in generator function 异步函数中的 while 循环抛出 StopIteration - While loop in async function throws StopIteration 读取 csv 数据时 pythoncode 中的 StopIteration 错误 - StopIteration Error in pythoncode while reading csv data 尝试为 model 构建数据输入时出现 StopIteration 错误 - StopIteration error while trying to build data input for a model 在Django setUp（）单元测试中使用factory_boy时出现“ StopIteration”异常 - “StopIteration” exception while using factory_boy in django setUp() unittest 使用 nltk 在 Python 3 中使用 next 时出现 StopIteration 错误 - StopIteration Error when using next in Python 3 using nltk function 中引发的 StopIteration 异常 - StopIteration exception raised in a function 使用列表时，Python3 StopIteration错误无法引发（map（...）） - Python3 StopIteration Error can not raise when using list(map(…))
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM