symfony3 inverse entity mapping slow

Question

I got the following entities:

<?php

namespace AppBundle\Entity;

use Doctrine\ORM\Mapping as ORM;
use Doctrine\Common\Collections\ArrayCollection;
use Doctrine\Common\Collections\Collection;

/**
 * Productnum
 *
 * @ORM\Table(name="productnum")
 * @ORM\Entity
 */
class Productnum 
{
  /**
   * @var object
   *
   * @ORM\OneToMany(
   *      targetEntity="AppBundle\Entity\Products",
   *      mappedBy="productnum",
   *      cascade={"persist", "remove"}
   * )
   */
  protected $productnumInverse;

  /**
   * Constructor
   */
  public function __construct()
  {
      $this->productnumInverse = new ArrayCollection();
  }


  /**
   * Get productnumInverse
   *
   * @return Collection
   */
  public function getProductnumInverse()
  {
      return $this->productnumInverse;
  }

}

<?php

namespace AppBundle\Entity;

use Doctrine\ORM\Mapping as ORM;
use Doctrine\Common\Collections\ArrayCollection;
use Doctrine\Common\Collections\Collection;


/**
 * Products
 *
 * @ORM\Table(name="products")
 * @ORM\Entity
 */
class Products
{
   /**
   * @var \AppBundle\Entity\Productnum
   *
   * @ORM\ManyToOne(targetEntity="AppBundle\Entity\Productnum", inversedBy="productnumInverse")
   * @ORM\JoinColumns({
   *   @ORM\JoinColumn(name="productnum_id", referencedColumnName="row_id")
   * })
   */
  public $productnum;

   /**
   * @var object
   *
   * @ORM\OneToMany(
   *      targetEntity="AppBundle\Entity\Product_region",  fetch="EAGER",
   *      mappedBy="productid",
   *      cascade={"persist", "remove"}
   * )
   */
  protected $productInverse;

  /**
   * Constructor
   */
  public function __construct()
  {
      $this->productInverse = new ArrayCollection();
  }
}

<?php

namespace AppBundle\Entity;

use AppBundle\AppBundle;
use AppBundle\Entity\Productnum_filial;
use Doctrine\ORM\Mapping as ORM;

/**
 * Productnum_region
 *
 * @ORM\Table(name="productnum_region")
 * @ORM\Entity
 */
class Productnum_region
{
  //regular getters and setters here...
}

And a mapping entity:

<?php

namespace AppBundle\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * Product_region
 *
 * @ORM\Table(name="product_region")
 * @ORM\Entity
 */
class Product_region
{
  /**
   * @ORM\Column(name="id", type="integer")
   * @ORM\Id
   * @ORM\GeneratedValue(strategy="IDENTITY")
   */
  private $id;

/**
 * @var \AppBundle\Entity\Products
 *
 * @ORM\ManyToOne(targetEntity="AppBundle\Entity\Products")
 * @ORM\JoinColumns({
 *   @ORM\JoinColumn(name="product_id", referencedColumnName="row_id")
 * })
 * 
 */
private $productid;

/**
 * @var \AppBundle\Entity\Productnum_region
 *
 * @ORM\ManyToOne(targetEntity="AppBundle\Entity\Productnum_region")
 * @ORM\JoinColumns({
 *   @ORM\JoinColumn(name="region_id", referencedColumnName="id")
 * })
 */
private $regionid;
}

In my controller I got the following code

$sql = sprintf("SELECT p FROM 'AppBundle:Productnum' p");
     $productnums = $em->createQuery($sql)
       ->setFirstResult($start)
       ->setMaxResults($length)
       ->getResult();
     $data = [];
     foreach($productnums as $productnum) {
         $prods = '';
         foreach($productnum->getProductnumInverse() as $product) {
            $filials = [];
            $regions = [];
            if($product && $product->getAllregions()){
                $regions[] = $filials[] = 'All';
                    } elseif($product && $product->getAllfilials()){
                      $filials[] = 'All';
                       $regs = $product->getProductInverse();
                       foreach($regs as $reg){        
                          $regions[] =  $reg->getRegionid()->getName();
                                }
                        }elseif($product){
                            $regs = $product->getProductInverse();
                            foreach($regs as $reg){
                                $fil = $reg->getRegionid()->getFilial()->getName();
                                if(!in_array($fil, $filials)){
                                    $filials[] = $fil;
                                }
                                $regions[] =  $reg->getRegionid()->getName();
                              }
  }
}

The problem is that on my local machine this code works fine, but on remote server it works very slow. I opened Symfony profiler both on my local machine and on server and saw that on my local machine (which was OK) it took 336 DB (1.5 sec total) queries to complete most of which were like the following;

SELECT t0.id AS id_1, t0.name AS name_2, t0.code AS code_3, t0.filial_id AS filial_id_4 FROM productnum_region t0 WHERE t0.id = ?
Parameters: [0 => 100] 

and

SELECT t0.row_id AS row_id_1, ...  t0.productnum_id AS productnum_id_21, t22.id AS id_23, t22.product_id AS product_id_24, t22.region_id AS region_id_25 FROM products t0 LEFT JOIN product_region t22 ON t22.product_id = t0.row_id WHERE t0.productnum_id = ?
Parameters: [0 => 945]

while on my server there were 36 queries in total (20 sec total, BAD), one of which (and probably the slowest one) was the following:

SELECT t0.row_id AS row_id_1, ... t0.productnum_id AS productnum_id_21, t22.id AS id_23, t22.product_id AS product_id_24, t22.region_id AS region_id_25 
FROM products t0 LEFT JOIN product_region t22 ON t22.product_id = t0.row_id WHERE t0.row_id IN (?)
Parameters: [ 0 => [ 0 => 2, 1 => 97, 2 => 212, 3 => 225, 4 => 297, 5 => 355, 6 => 356, 7 => 482, 8 => 571, 9 => 737, 10 => 789 
...MANY MANY MANY data here...

So the question is how could happen that the same code on different machines would convert to different queries and how this could be avoided?

Thank you

Answer 1

It does look like you may have an overall mistake with your database design or code that is requiring you to have 4 foreach loops and 2 if statements all nested together.

To answer your question specifically - you need to join your corresponding entities in your query - Doctrine isn't going to do it for you. So when you perform this:

foreach($productnum->getProductnumInverse() as $product) {

Every iteration through that loop, Doctrine is going to individually query that $product , because it didn't select it in your original query. That is why you see 336 database queries, when you really should only see one. Instead of:

SELECT p FROM 'AppBundle:Productnum' p

your query should look more like this:

SELECT p, pi, pip, pir
FROM AppBundle:Productnum p
JOIN p.productnumInverse pi
JOIN pi.product pip
JOIN pi.region pir

This should drastically reduce the number of queries you are running - ideally you should be down to 1 query for retrieving all that data. In short, Doctrine doesn't join associated entities unless you specifically tell it to.