How to optimze insert in CodeIgniter (insert faster)

Question

I want to create a table with all the links in another one, but the proccess is too slow, I am doing this with CodeIgniter, and I want to know if there is a better way to do it.

This is my code:

foreach($results as $value) { 
        $ci->db->select('id');
        $ci->db->where('url', $value->Url);
        $link = $ci->db->get('links')->row();

        if(@!$link) {
            $ci->db->insert('links', array(
                'title' => $value->Title,
                'url' => $value->Url,
                'description' => $value->Description,
                'is_featured' => 0,
                'published_date' => date('Y-m-d h:i:s')
            ));
            $link_id = $ci->db->insert_id();
        } else {
            $link_id = $link->id;
        }


        $ci->db->where('link_id', $link_id);
        $ci->db->where('term_id', $term_id);
        $term_links = $ci->db->get('term_links')->row();

        if(!$term_links) {
            $ci->db->insert('term_links', array(
                'link_id'=>$link_id, 
                'term_id'=>$term_id,
                'order_link'=>$order,
                'duplicates'=>0
            ));
        } else {
            $ci->db->where('id', $term_links->id);
            $ci->db->update('term_links', array('duplicates'=>$term_links->duplicates+=1));
        }

        $order++;
    }

Any idea? I use activerecords instead SQL.

Thank you in advance!

Answer 1

The fastest mechanism is to write out a text file CSV and perform a LOAD DATA INFILE. Often it goes into a worktable that is cleared first. Then that table receives a call for insert realTable (col1,col2,etc) select col1,col2,etc from workTable.

This is hands down the fastest way.

So there are 2 calls. If that second call is an insert with join pattern, that is fine.

But to loop will always be the slow road.

Using this strategy will vastly out perform looping.

CI can perform this via raw queries. So any struggle with the ORM is easily circumvented.

Answer 2

Well you are not executing a single insert statement you are are executing many insert and update statements in a loop. Consider changing your code to user insert_batch

Insert batch generates the query and runs it all at once instead of multiple individual queries. That speeds things up in two ways. First: There is only one back and forth communication between PHP and the DB instead of several. Second: index updates happen only once instead of several times.