some records not update with bulk update in PHP and MySql

Question

i have a php script for bulk update, in my database exists 90,585 rows but only update 85,282 rows, and i don't know why, this is my script:

//limit the select to 100
//Bulk update is very slow, is you set a limit very high server crash
$limit = 100;
//$messages = array();

$updated_posts = 0;

//maybe there are better solutions for this
for ($i=0;$i<=$totalMsg;$i+=$limit)
{

    $get_posts = mysqli_query($conn,
    "SELECT id_msg, body
    FROM " . $to_prefix ."messages
    WHERE id_msg != 0
    LIMIT " . $i ."," . $limit);

    //The post Array
    $messages = array();
    while($row = mysqli_fetch_assoc($get_posts))
    {
        $messages[$row['id_msg']] = array(
            'body' => fixBBCTags($row['body']),
        );
    }

    //update data!, good luck!!
    bulkUpdate($conn,$to_prefix."messages","id_msg","body",$messages);

    $updated_posts += mysqli_affected_rows($conn);

}// for loop

this is the bulkUpdate() function:

function bulkUpdate($conn, $table, $id_column, $update_column, array &$idstovals)
{
global $error; 

//prepare the Bulk Update SQL
$sql = "update " . $table . " set " . $update_column . " = CASE " . $id_column ;

foreach($idstovals as $id => $val)
{
    $sql .= " WHEN " . "'" . $id . "'" . " THEN " . "'" . mysqli_real_escape_string($conn,$val[$update_column]) . "'" . " \n";
}

$sql .= " END 
WHERE " . $id_column. " in (" . implode(',', array_keys($idstovals)) . ")";

//reset the array
//$idstovals=array();

//try update the bulk
$update = mysqli_query($conn,$sql);

if(mysqli_error($conn))
    $error = mysqli_error($conn);
}

All the rows must be updated, there are better solution for this?

Regards.

Answer 1

When you wrote the same data in a row, MySQL do not write this record

MariaDB [test]> create table r (id integer, PRIMARY KEY (id) );
Query OK, 0 rows affected (0.15 sec)

MariaDB [test]> insert into r (id)  VALUES (1),(2),(3);
Query OK, 3 rows affected (0.00 sec)
Records: 3  Duplicates: 0  Warnings: 0

MariaDB [test]> update r set id=4 where id=1;
Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0

MariaDB [test]> update r set id=4 where id=4;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 1  Changed: 0  Warnings: 0

Answer 2

Here my second try,

UPDATE updatetable u 
LEFT JOIN reftable r ON r.id = p.id
SET u.updatecol = r.refcol
WHERE r.id_msg !=0 ;

Or you build a array in PHP with id (PRIMARY KEY) and new values. So you can generate a bulk INSERT with ON DUPLICATE KEY UPDATE like this

INSERT INTO mytable (id,col) VALUES
(1,'new val'),
(2,'new val'),
(3,'new val'),
(66,'new val'),
...
(80000,'new val for id 80000')
ON DUPLICATE KEY UPDATE col= VALUES(col);

Answer 3

Sorry am late, here my Answer. Here is a Sample. I h dont test it. It is importent that the Column id_msg is the PRIMARY KEY or have a unique INDEX.

//limit the select to 100
//Bulk update is very slow, is you set a limit very high server crash
$limit = 100;
$messages = array();

$updated_posts = 0;

$get_posts = mysqli_query($conn,
"SELECT id_msg, body FROM " . $to_prefix ."messages WHERE id_msg != 0");

while($row = mysqli_fetch_assoc($get_posts))
{
    $messages[] = "('".$row['id_msg'] ."','"
                    .mysqli_real_escape_string(fixBBCTags($row['body'])) ."')";
}

$sql_pre  = "INSERT INTO ".$to_prefix.".messages (id_msg,message) VALUES "
$sql_post = " ON DUPLICATE KEY message = VALUE(message)"

$sql = "";
$sep = "";
$cnt = 0;


foreach($messages as $msg)
{
    $sql = $sep . $msg;
    $sep = ",";

    if ($cnt++ % $limit) {
        mysqli_query($conn,$sql_pre . $sql . $sql_post);
        $pre = "";
    }
}

// Write the Rest if one
if ( $sql <> "" ) {
    mysqli_query($conn,$sql_pre . $sql . $sql_post);    
}