How to import csv file to mysql more faster codeigniter

Question

Its working but when I import big files almost 2500+ lines in CSV files it load/import slowly and its say's error but some files are inserting ,some are not. And my question is there way to make it faster importing? I see in file But i don't get it how to use it.

Code:

    function convert_csv($id){
                if (empty($id)) redirect("contracts");
                $config['upload_path']   = './files/contracts-csv/'; 
                $config['allowed_types'] = 'csv';   
                $config['max_size']      = '4096';      
                $config['overwrite']     =  TRUE;

                $this->load->library('upload', $config);
                $this->upload->display_errors('', '');

                if (!$this->upload->do_upload("csv_file")) {
                      echo $this->upload->display_errors(); die();
                      $this->data['error'] = array('error' => $this->upload->display_errors());
                } else {
                    $upload_result = $this->upload->data();
                    $upload_result = $this->upload->data(); 
                    $this->load->library('csvreader');
                    $result =   $this->csvreader->parse_file("./files/contracts-csv/".$upload_result['file_name']);
                    $date_today = date('Y-m-d H:i:s');
                        foreach ($result as $v) {
                            $this->contract_items->add_serial($v,$id,$date_today);
                        }
                }
}

Answer 1

Use MySQL transactions in CodeIgniter , it will speed up everything a lot (eg. make chunks of 500 items and insert them in one transaction).

But if it ends with error it looks like the problem might be somewhere else. 2500 isn't that much.

Answer 2

each iteration for line in csv does the next things:

• build insert sql
• send it to db layout
• get response

you can try to build bulk sql like:

insert into table_name (f1, f2, f3) values ("val1", "val2", "val3"), ("val2_1", "val2_2", "val2_3"),...

about 100-200 '("val1", "val2", "val3")' blocks it should be faster, but I dont know how about codeigniter cache and other things...

Answer 3

You could wrap it up in an insert_batch query.

$your_data = array();
foreach ($result as $v) {
    $your_data[] = array(
        'v' => $v,
        'id' => $id,
        'date_today' => $today,
    );
);

$this->db->insert_batch('your_table_name', $your_data);

This will run the entire thing as a single query. Which should be faster than multiple queries. The exact code is up to you, as I don't know $this->contract_items->add_serial does.