[英]umongo, pymongo, python 3, how do i load data from reference field(s)
I'm trying to understand how and why it's so hard to load my referenced data, in unmongo/pymongo我试图了解如何以及为什么在 unmongo/pymongo 中加载我的引用数据如此困难
@instance.register
class MyEntity(Document):
account = fields.ReferenceField('Account', required=True)
date = fields.DateTimeField(
default=lambda: datetime.utcnow(),
allow_none=False
)
positions = fields.ListField(fields.ReferenceField('Position'))
targets = fields.ListField(fields.ReferenceField('Target'))
class Meta:
collection = db.myentity
when i retrieve this with:当我用以下方法检索它时:
def find_all(self):
items = self._repo.find_all(
{
'user_id': self._user_id
}
)
return items
and then dump it like so:然后像这样转储它:
from bson.json_util import dumps
all_items = []
for item in all_items:
all_items.append(item.dump())
return dumps(all_items)
i get the following JSON object:我得到以下 JSON object:
[
{
"account": "5e990db75f22b6b45d3ce814",
"positions": [
"5e9a594373e07613b358bdbb",
"5e9a594373e07613b358bdbe",
"5e9a594373e07613b358bdc1"
],
"date": "2020-04-18T01:34:59.919000+00:00",
"id": "5e9a594373e07613b358bdcb",
"targets": [
"5e9a594373e07613b358bdc4",
"5e9a594373e07613b358bdc7",
"5e9a594373e07613b358bdca"
]
}
]
and without dump
并且没有
dump
<object Document models.myentity.schema.MyEntity({
'targets':
<object umongo.data_objects.List([
<object umongo.frameworks.pymongo.PyMongoReference(
document=Target,
pk=ObjectId('5e9a594373e07613b358bdc4')
)>,
<object umongo.frameworks.pymongo.PyMongoReference(
document=Target,
pk=ObjectId('5e9a594373e07613b358bdc7')
)>,
<object umongo.frameworks.pymongo.PyMongoReference(
document=Target,
pk=ObjectId('5e9a594373e07613b358bdca'))>]
)>,
'id': ObjectId('5e9a594373e07613b358bdcb'),
'positions':
<object umongo.data_objects.List([
<object umongo.frameworks.pymongo.PyMongoReference(
document=Position,
pk=ObjectId('5e9a594373e07613b358bdbb')
)>,
<object umongo.frameworks.pymongo.PyMongoReference(
document=Position,
pk=ObjectId('5e9a594373e07613b358bdbe'))>,
<object umongo.frameworks.pymongo.PyMongoReference(
document=Position,
pk=ObjectId('5e9a594373e07613b358bdc1'))>])>,
'date': datetime.datetime(2020, 4, 18, 1, 34, 59, 919000),
'account': <object umongo.frameworks.pymongo.PyMongoReference(document=Account, pk=ObjectId('5e990db75f22b6b45d3ce814'))>
})>
ie what if there's a reference field in 'target' as well?即如果“目标”中也有参考字段怎么办? I understand this can be expensive on the DB, but is there some way to specify this on the schema definition itself?
我知道这在数据库上可能很昂贵,但是有没有办法在架构定义本身上指定它? ie in meta class, that i always want the full, dereferenced object for a particular field?
即在元 class 中,我总是想要特定字段的完整、取消引用的 object?
I'm not that new to mongo, but new to python / mongo.我对 mongo 并不陌生,但对 python / mongo 并不陌生。 I feel like I'm missing something fundamental here.
我觉得我在这里缺少一些基本的东西。
EDIT: so right after posting, i did find this issue:编辑:所以在发布后,我确实发现了这个问题:
https://github.com/Scille/umongo/issues/42 https://github.com/Scille/umongo/issues/42
which provides a way forward这提供了前进的道路
is this still the best approach?这仍然是最好的方法吗? Still trying to understand why this is treated like an edge case.
仍在试图理解为什么这被视为边缘案例。
EDIT 2: progress编辑2:进展
class MyEntity(Document):
account = fields.ReferenceField('Account', required=True, dump=lambda: 'fetch_account')
date = fields.DateTimeField(
default=lambda: datetime.utcnow(),
allow_none=False
)
#trade = fields.DictField()
positions = fields.ListField(fields.ReferenceField('Position'))
targets = fields.ListField(fields.ReferenceField('Target'))
class Meta:
collection = db.trade
@property
def fetch_account(self):
return self.account.fetch()
so with the newly defined property decorator, i can do:所以使用新定义的属性装饰器,我可以做到:
items = MyEntityService().find_all()
allItems = []
for item in allItems:
account = item.fetch_account
log(account.dump())
allItems.append(item.dump())
When I dump account, all is good.当我转储帐户时,一切都很好。 But I don't want to explicitly/manually have to do this.
但我不想明确/手动必须这样做。 It still means I have to recursively unpack and then repack each referenced doc, and any child references, each time I make a query.
这仍然意味着我每次进行查询时都必须递归地解包然后重新打包每个引用的文档和任何子引用。 It also means the schema SOT is no longer contained just in the umongo class, ie if a field changes, I'll have to refactor every query that uses that field.
这也意味着模式 SOT 不再仅包含在 umongo class 中,即如果字段更改,我将不得不重构使用该字段的每个查询。
I'm still looking for a way to decorate/flag this on the schema itself.我仍在寻找一种在架构本身上装饰/标记它的方法。 eg
例如
account = fields.ReferenceField('Account', required=True, dump=lambda: 'fetch_account')
dump=lambda: 'fetch_account'
i just made up, it doesn't do anything, but that's more or less the pattern i'm going for, not sure if this is possible (or even smart: other direction, pointers to why i'm totally wrong in my approach are welcome).... dump=lambda: 'fetch_account'
我只是编造的,它什么也没做,但这或多或少是我想要的模式,不确定这是否可能(或者甚至聪明:其他方向,指向为什么我欢迎我的方法完全错误)....
EDIT 3: so this is where i've landed:编辑3:所以这就是我登陆的地方:
@property
def fetch_account(self):
return self.account.fetch().dump()
@property
def fetch_targets(self):
targets_list = []
for target in self.targets:
doc = target.fetch().dump()
targets_list.append(doc)
return targets_list
@property
def fetch_positions(self):
positions_list = []
for position in self.positions:
doc = position.fetch().dump()
positions_list.append(doc)
return positions_list
and then to access:然后访问:
allItems = []
for item in items:
account = item.fetch_account
positions = item.fetch_positions
targets = item.fetch_targets
item = item.dump()
item['account'] = account
item['positions'] = positions
item['targets'] = targets
# del item['targets']
allTrades.append(item)
I could clean it up/abstract it some, but i don't see how i could really reduce the general verbosity at at this point.我可以将其清理/抽象一些,但我不知道此时我如何才能真正减少一般冗长。 It does seem to be give me the result i'm looking for though:
它似乎确实给了我我正在寻找的结果:
[
{
"date": "2020-04-18T01:34:59.919000+00:00",
"targets": [
{
"con_id": 331641614,
"value": 106,
"date": "2020-04-18T01:34:59.834000+00:00",
"account": "5e990db75f22b6b45d3ce814",
"id": "5e9a594373e07613b358bdc4"
},
{
"con_id": 303019419,
"value": 0,
"date": "2020-04-18T01:34:59.867000+00:00",
"account": "5e990db75f22b6b45d3ce814",
"id": "5e9a594373e07613b358bdc7"
},
{
"con_id": 15547841,
"value": 9,
"date": "2020-04-18T01:34:59.912000+00:00",
"account": "5e990db75f22b6b45d3ce814",
"id": "5e9a594373e07613b358bdca"
}
],
"account": {
"user_name": "hello",
"account_type": "LIVE",
"id": "5e990db75f22b6b45d3ce814",
"user_id": "U3621607"
},
"positions": [
{
"con_id": 331641614,
"value": 104,
"date": "2020-04-18T01:34:59.728000+00:00",
"account": "5e990db75f22b6b45d3ce814",
"id": "5e9a594373e07613b358bdbb"
},
{
"con_id": 303019419,
"value": 0,
"date": "2020-04-18T01:34:59.764000+00:00",
"account": "5e990db75f22b6b45d3ce814",
"id": "5e9a594373e07613b358bdbe"
},
{
"con_id": 15547841,
"value": 8,
"date": "2020-04-18T01:34:59.797000+00:00",
"account": "5e990db75f22b6b45d3ce814",
"id": "5e9a594373e07613b358bdc1"
}
],
"id": "5e9a594373e07613b35
8bdcb"
}
]
It seems like this is a design choice in umongo.这似乎是 umongo 中的一种设计选择。
In Mongoid for example (the Ruby ODM for MongoDB), when an object is referenced it is fetched from the database automatically through associations as needed.例如,在 Mongoid(MongoDB 的 Ruby ODM)中,当引用 object 时,它会根据需要通过关联自动从数据库中获取。
As an aside, in an ODM the features of "define a field structure" and "seamlessly access data through application objects" are quite separate.顺便说一句,在 ODM 中,“定义字段结构”和“通过应用程序对象无缝访问数据”的特性是完全不同的。 For example, my experience with Hibernate in Java suggests it is similar to what you are discovering with umongo - once the data is loaded, it provides a way of accessing the data using application-defined field structure with types etc., but it doesn't really help with loading the data from application domain transparently.
例如,我在 Java 中使用 Hibernate 的经验表明它与您在 umongo 中发现的类似 - 一旦加载数据,它提供了一种使用应用程序定义的字段结构和类型等访问数据的方法,但它没有t 真正有助于透明地从应用程序域加载数据。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.