[英]Django - Following A Foreign Key To Another Foreign Key
I'm trying to speedup lookups on my models. 我正在尝试加快模型的查找速度。 I have 3 models. 我有3个模型。
class sett(models.Model):
setid = models.IntegerField(primary_key=True)
block = models.ForeignKey(mapt, related_name='sett_block')
class data_parsed(models.Model):
setid = models.ForeignKey(sett, related_name='data_parsed_setid', primary_key=True, db_index = True)
setid_hash = models.CharField(max_length=100, db_index = True)
class unique_variables(models.Model):
setid = models.ForeignKey(sett, related_name='unique_variables_setid', db_index = True)
setid_hash = models.CharField(max_length=100, db_index = True)
Most importantly, sett has a primary key called setid, which: 最重要的是,sett有一个称为setid的主键,该主键为:
data_parsed, and unique_variables have a foreign key to setid data_parsed和unique_variables具有要设置的外键
data_parsed: 3 million rows data_parsed:300万行
unique_variables: 13 million rows unique_variables:1300万行
Then I have some piece of code that tries to iterate through all data_parsed objects. 然后,我有一些代码试图遍历所有data_parsed对象。
def something_1():
timeNow = datetime.datetime.now()
for i in range(0,N) <---- N is predefined, and no less than 10
for contents in data_parsed.objects.all().iterator():
pass
timeEnd = datetime.datetime.now()
print str(timeEnd - timeNow)
data_parsed has a few million items, and it takes around 20 seconds to go through, which I think is pretty fast. data_parsed有几百万个项目,大约需要20秒才能通过,我认为这非常快。
Now, since setid is the same fields on both data_parsed, and unique_variables. 现在,由于setid是data_parsed和unique_variables上的相同字段。 I can get items in unique_variables with the foriegn key of data_parsed. 我可以使用data_parsed的foriegn键获取unique_variables中的项目。 Hence: 因此:
def something_2():
timeNow = datetime.datetime.now()
for i in range (0, N) <---- N is predefined, and no less than 10
for contents in data_parsed.objects.all().iterator():
try:
items = unique_variables.objects.get(setid = contents.setid)
except:
pass
timeEnd = datetime.datetime.now()
print str(timeEnd - timeNow)
However, the problem is that this suddenly consumes hours of time. 但是,问题在于这突然消耗了数小时的时间。 Since I cannot use select_related, since I'm not going back to sett, but actually moving from sett to unique_variables. 由于我无法使用select_related,因此由于我不打算返回sett,而是实际上从sett迁移到unique_variables。
Is there a way to speed up getting items with this behavior? 有没有一种方法可以加快这种行为的获取速度?
Yes, you can use prefetch_related . 是的,您可以使用prefetch_related 。 I've linked the docs for you. 我已经为您链接了文档。
def something_2():
timeNow = datetime.datetime.now()
for i in range (0, N) <---- N is predefined, and no less than 10
parsed_objects = data_parsed.objects.all().prefetch_related(
'setid__unique_variables_setid')
for contents in parsed_objects:
try:
items = list(contents.setid.unique_variables_setid.all())
except:
pass
timeEnd = datetime.datetime.now()
print str(timeEnd - timeNow)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.