[英]Pagerfanta and Doctrine2 COUNT optimization
I'm using Pagerfanta and Doctrine Adapters with Symfony2 and Silex. 我正在使用Pagerfanta和Doctrine Adapters与Symfony2和Silex。 As my database became bigger I noticed huge load on admin stats pages that display big data with pagination.
随着我的数据库变得越来越大,我发现管理员统计信息页面上的大量负载会显示带有分页的大数据。 I checked profiler and saw unbelievably inefficient queries:
我检查了分析器,看到了令人难以置信的低效查询:
SELECT DISTINCT id16
FROM (
SELECT f0_.username AS username0, ..., f0_.added_on AS added_on20
FROM fos_user f0_ ORDER BY f0_.id DESC
) dctrn_result
LIMIT 50 OFFSET 0;
SELECT COUNT(*) AS dctrn_count
FROM (
SELECT f0_.username AS username0, ..., f0_.added_on AS added_on20
FROM fos_user f0_ ORDER BY f0_.id DESC
) dctrn_result
LIMIT 50 OFFSET 0;`
First query was easy to fix by creating fixed version of DoctrineORMAdapter
class. 通过创建
DoctrineORMAdapter
类的固定版本,可以轻松修复第一个查询。 The code that generates COUNT()
query is more complicated so I decided to ask if there's any solution for this. 生成
COUNT()
查询的代码更复杂,所以我决定询问是否有任何解决方案。
So is there any way to make Pagerfanta not running nested queries? 那么有没有办法让Pagerfanta不运行嵌套查询?
Better late than never: I've hit the same wall today with >200k records and found a solution. 迟到总比没有好:我今天在同一面墙上打了超过20万的记录并找到了解决方案。
Pagerfanta internally uses Doctrine\\ORM\\Tools\\Pagination\\CountOutputWalker to count objects which results in a count query like this: Pagerfanta内部使用Doctrine \\ ORM \\ Tools \\ Pagination \\ CountOutputWalker来计算导致计数查询的对象,如下所示:
SELECT
COUNT(*) AS dctrn_count
FROM
(
SELECT
DISTINCT id_0
FROM
(
SELECT
m0_.id AS id_0,
...
FROM
messaging_messages m0_
ORDER BY
m0_.id DESC
) dctrn_result
) dctrn_table
To bypass CountOutputWalker we can pass a flag when instantiating DoctrineORMAdapter. 为了绕过CountOutputWalker,我们可以在实例化DoctrineORMAdapter时传递一个标志。 So instead of simply
而不是简单的
$adapter = new DoctrineORMAdapter($qb);
you do 你做
$adapter = new DoctrineORMAdapter($qb, true, false);
(third parameter). (第三个参数)。 This turns the count query into a much more efficient one:
这会将计数查询转换为更有效的查询:
SELECT
count(DISTINCT m0_.id) AS sclr_0
FROM
messaging_messages m0_
You have to update whiteoctober/Pagerfanta to 1.0.3 though. 你必须将whiteoctober / Pagerfanta更新为1.0.3。
In your case it's not pagerfanta that does the sub-queries. 在你的情况下,不是pagerfanta进行子查询。 It's the source your query builder instance it's coming from.
它是您的查询构建器实例的来源。
I usually have a function in entity repository that returns a plain query builder instance instead of the results. 我通常在实体存储库中有一个函数,它返回一个普通的查询构建器实例而不是结果。 It's left to you to write an efficient query builder.
您可以编写一个高效的查询构建器。 Then I feed that query builder into DoctrineORMAdapter.
然后我将该查询构建器提供给DoctrineORMAdapter。
I've got this helper function that I use throughout my projects: 我有我在整个项目中使用的辅助函数:
/**
* Pass an array, entity or a custom QueryBuilder instance to paginate.
* Takes an array of parameters as a second argument.
* Default parameter values:
*
* $params = array(
* 'curPage' => 1,
* 'perPage' => 15,
* 'order' => 'DESC'
* );
*
* @param mixed $object
* @param array $params
*
* @return Pagerfanta
*/
public function paginate($object, $params = array())
{
if (is_array($object)) {
$adapter = new ArrayAdapter($object);
} elseif ($this->isEntity($object)) {
$qb = $this->em->createQueryBuilder()
->select('s')
->from($this->getEntityName($object), 's')
->orderBy('s.id', isset($params['order']) ? $params['order'] : 'DESC');
$adapter = new DoctrineORMAdapter($qb);
} elseif ($object instanceof QueryBuilder) {
$adapter = new DoctrineORMAdapter($object);
}
$pager = new Pagerfanta($adapter);
$pager->setMaxPerPage(isset($params['perPage']) ? $params['perPage'] : 15);
$pager->setCurrentPage(isset($params['curPage']) ? $params['curPage'] : 1);
return $pager;
}
You can pass an array, entity or a query builder instance and it will return an appropriately paginated object ready to use. 您可以传递数组,实体或查询构建器实例,它将返回一个准备好使用的适当分页对象。
You probably know how it's done, but anyway, here's what I have in my entity repository - one function returns query builder instance (perfect for pagerfanta), the other returns an array to be used elsewhere: 您可能知道它是如何完成的,但无论如何,这是我在实体存储库中的内容 - 一个函数返回查询构建器实例(适用于pagerfanta),另一个返回一个要在别处使用的数组:
public function getMessageQueryBuilder($campaignId, $eqCriteriaArray = array(), $neqCriteriaArray = array())
{
$qb = $this->createQueryBuilder('m');
$qb->select('m')
->leftJoin('m.campaign', 'c')
->leftJoin('m.sentBy', 'u')
->where($qb->expr()->eq('m.campaign', $campaignId));
foreach ($eqCriteriaArray as $property => $value) {
$qb->andWhere($qb->expr()->eq($property, $qb->expr()->literal($value)));
}
foreach ($neqCriteriaArray as $property => $value) {
$qb->andWhere($qb->expr()->neq($property, $qb->expr()->literal($value)));
}
return $qb->orderBy('m.id', 'DESC');
}
public function filterMessages($campaignId, $eqCriteriaArray = array(), $neqCriteriaArray = array())
{
return $this->getMessageQueryBuilder($campaignId, $eqCriteriaArray, $neqCriteriaArray)->getQuery()->getResult();
Then I combine those two to get the actual pager object: 然后我将这两个结合起来得到实际的寻呼机对象:
$singleSmsPager = $this->pagerUtil->paginate(
$this->em->getRepository('TreasureForgeMessageBundle:Message')
->getMessageQueryBuilder(CcToolSender::CAMPAIGN_ID, array(), array('u.username' => 'admin')),
array(
'curPage' => $singleSmsPage,
'perPage' => 10
)
);
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.