[英]Looping a PYTHON script
I need some help with Python loop techniques. 我需要Python循环技术方面的帮助。 After a few days of searching, I give up...
经过几天的搜寻,我放弃了...
System: Windows ( Anaconda ) 系统:Windows(Anaconda)
Idea: "I created an HTML parser script, but due to missing knowledge and experience with Python scripts, it appears that I need to run it on every page. I can't fix it, that's why I decided to loop this script and make it run 100 times for 100 pages"....but as a result, I can't find the right way to do so... 想法:“我创建了HTML解析器脚本,但是由于缺少Python脚本的知识和经验,看来我需要在每个页面上运行它。我无法修复它,这就是为什么我决定循环该脚本并制作一个它可以运行100次,每次100页”。...但是,结果是,我找不到正确的方法……
My script 我的剧本
import requests
import pandas as pd
import urllib.parse
import urllib.request
import re
import os
import sys
url = "*******************/store/index.php"
querystring ={"id":"***","act":"search","***":"***","country":"",
"state":"*","city":"","zip":"","type":"","base":"","PAGENUM":"2"}
headers = {
'Host': "www.*****",
'Connection': "keep-alive",
'Upgrade-Insecure-Requests': "1",
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/64.0.3282.119 Safari/537.36",'Accept':"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
'Referer': "h************/store/index.php?id=********************&pagenum=2",
'Accept-Encoding': "gzip, deflate",
'Accept-Language': "en-US,en;q=0.9",
'Cookie': "php_session_id_real=**********; cookname=**********; cook******",
'cache-control': "no-cache",
'Postman-Token': "**************************"
}
response = requests.request("GET", url, headers=headers,params=querystring)
df_list = pd.read_html(response.text)
df = df_list[-1]
print(df)
All I need to change is PAGENUM querystring ( ex: &pagenum=2,3,10,50 ,etc ... ) 我需要更改的只是PAGENUM querystring(例如: &pagenum = 2,3,10,50等)
Is it possible to run this python script X times, and each time change the value of pagenum = pagenum + 1 ?? 是否可以运行此python脚本X次,并且每次更改pagenum = pagenum + 1的值?
Hope for your advice! 希望您的指教!
Cheers 干杯
Use a for
and iterate to a list containing all the desired values. 使用
for
并迭代到包含所有所需值的列表。 Next, use str
to store the value in the dictionary. 接下来,使用
str
将值存储在字典中。
Do this: 做这个:
import requests
import pandas as pd
import urllib.parse
import urllib.request
import re
import os
import sys
pagenums=[2,3,10,50]
#or pagenums = np.range(1,101)
for page in pagenums:
querystring ={"id":"***","act":"search","***":"***","country":"",
"state":"*","city":"","zip":"","type":"","base":"","PAGENUM":str(page)}
#......
#..... # more code here
#headers = {....}
For each iteration, the value of PAGENUM
key, will updated. 对于每次迭代,
PAGENUM
键的值都会更新。
You need to apply for loop which runs 100 times and takes all your pages. 您需要申请运行100次并占用您所有页面的循环。 I hope below code works fine.
我希望下面的代码能正常工作。
import requests
import pandas as pd
import urllib.parse
import urllib.request
import re
import os
import sys
import numpy as np
url = "*******************/store/index.php"
pagenums = np.arange(0,100)
for i in pagenums:
querystring ={"id":"***","act":"search","***":"***","country":"",
"state":"*","city":"","zip":"","type":"","base":"","PAGENUM":str(i)}
headers = {
'Host': "www.*****",
'Connection': "keep-alive",
'Upgrade-Insecure-Requests': "1",
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/64.0.3282.119 Safari/537.36",'Accept':"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
'Referer': "h************/store/index.php?id=********************&pagenum=2",
'Accept-Encoding': "gzip, deflate",
'Accept-Language': "en-US,en;q=0.9",
'Cookie': "php_session_id_real=**********; cookname=**********; cook******",
'cache-control': "no-cache",
'Postman-Token': "**************************"
}
response = requests.request("GET", url, headers=headers,params=querystring)
df_list = pd.read_html(response.text)
df = df_list[-1]
enter code here
print(df)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.