[英]Faster way to do mysql query in python
There are list1 and list2, each contains 1,104,824 values 有list1和list2,每个包含1,104,824个值
table1 has 350,000,000 rows with 3 columns: ID, name1, name2 table1具有350,000,000行,其中包含3列:ID,name1,name2
and this is what I tried to do: 这就是我试图做的:
con = mdb.connect('localhost','user','password','db')
cur = con.cursor()
for i in range(1104824)
sql ="select count(distinct(a.ID)) from (select name1 ,ID from table1 where name2 <> '"+str(list1[i])+"') as a where a.name1 = '"+str(list2[i])+"'"
cur.execute(sql)
data = cur.fetchone()[0]
but it is very very slow. 但是它非常非常慢。 Is there any faster way to do this query?
有没有更快的方法来执行此查询?
This is your query: 这是您的查询:
select count(distinct a.ID)
from (select name1, ID
from table1
where name2 <> '"+str(list1[i])+"'
) a
where a.name1 = '"+str(list2[i])+"'";
I would recommend writing this as: 我建议将其写为:
select count(distinct ID)
from table1
where name2 <> '"+str(list1[i])+"' and
name1 = '"+str(list2[i])+"'";
Then you can speed up the query with an index on table1(name1, name2, id)
-- all three columns in that order. 然后,您可以使用
table1(name1, name2, id)
上的索引来加快查询速度- table1(name1, name2, id)
顺序的所有三列。
Note: I would write the sql as: 注意:我将sql编写为:
sql = """
select count(distinct ID)
from table1
where name2 <> '{0}' and name1 = '{1}'
""".format(str(list1[i]), str(list2[i]))
Seems like this would work as well with the appropriate indices: 似乎这样与适当的索引也可以工作:
select count(distinct id)
from table1
where name2 <> 'Name1'
and name1 = 'Name2'
Look into using parameterized queries though. 不过请注意使用参数化查询。 Your query is vulnerable to sql injection and would break for names with apostrophes for example... Lots of examples out there, here's a couple: Python MySQL Parameterized Queries and https://stackoverflow.com/a/1633589/1073631
您的查询容易受到sql注入的攻击,例如,可能会打乱带有单引号的名称...那里有很多示例,下面是几个示例: Python MySQL参数化查询和https://stackoverflow.com/a/1633589/1073631
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.