performance of Entity manager on Symfony

Question

I have to make a command for data migration but the time execution is too long and crash after a long time, i need to make it faster, can you help me ? Here an example :

1. Situation

   • I have a Matter Entity
     
   • I have a Customer Entity
     
   • I have a MatterCustomer Entity
     
   • Matter have One Or Many MatterCustomer
     
   • Customers can be in None Or Many MatterCustomer
     Note that the link between entities are correctly set
     

2. Command

    function execute(InputInterface $input, OutputInterface $output) { // Here connection to old database $this->pdo = $this->container()->get('app.command_oldbase.odbase_helper')->connectToDatabase(); $em = $this->getContainer()->get('doctrine')->getManager(); $em->getConnection()->beginTransaction() $em->getConnection()->setAutoCommit(false); try{ $this->migrateMatters($output); } catch (Exception $e) { $em->getConnection()->rollback(); throw $e; } $em->getConnection()->commit(); } function migrateMatters(OutputInterface $output) { $statement = $this->pdo->prepare('Select id,name From oldmatter'); $statement->execute(); $oldmatters = $statement->fetchAll(); foreach ($oldmatters As $oldmatter) { $this->em = $this->getContainer()->get('doctrine')->getManager(); $matter = (new Matter()) ->setName($oldmatter['name']) ; $this->em->persist($matter); $this->em->flush(); $this->setMatterCustomer($matter, $oldmatter['id']); $this->em->clear(); } } function setMatterCustomer(Matter $matter, int $oldmatter_id) { $statement = $this->pdo->prepare('Select customername From oldmattercustomers Where oldmatter_id = :oldmatter_id'); $statement->bindParam(':oldmatter_id', $oldmatter_id); $statement->execute(); $oldcustomers = $statement->fetchAll(); foreach ($oldcustomers As $oldcustomer) { //search customer in Customer entity $customer = $this->em->getRepository(Customer::class)->findOneBy(['name' => $oldcustomer['customername']]); //search if MatterCustomer exist $matterCustomer = $this->em->getRepository(MatterCustomer::class)->findOneBy(['matter' => $matter, 'customer' => $customer]); if ($matterCustomer == null) { $matterCustomer = (new MatterCustomer()) ->setContact($contact) ->setMatter($matter) ; //add MatterCustomer to arrayCollection in Matter $matter->addCustomer($matterCustomer); $this->em->persist($matter); $this->em->flush(); } } } 

Answer 1

1. Use batch processing

2. Use cascade persist for Matter customer. So, first you need create Matter instance, then add to than instance MatterCustomer and so on. And then perform persisting only Matter instance of course.

3. If there is a number of entities, probably disabling logging will reduce memory consumption

$this->em->getConnection()->getConfiguration()->setSQLLogger(null);

4. Run the command with --no-debug option (or in prod mode), it will reduce memory consumption.