仅将新记录从一个数据库服务器插入另一个数据库

Question

We have a phone system database on one server but we want an updated clone on our production server.

I've dumped the original and imported it to our local production server, which is great, but now I have to go about keeping it updated with any daily changes on the other server.

I'm trying to create a php script that will simply check each table on the other server and then insert any new records in each table into the cloned table on our production database.

Below is a test select/insert block. The select query works with no errors but my insert returns an error as I expected, because I believe I'm using NOT EXISTS incorrectly. I don't know if that's even the best route to take.

The session table has about 35 columns so I'm looking for the best way to go about this without having to declare every column if possible. The only thing I though of is the SESSION.SESSIONID field which is the primary key. So my logic below was, if the sessionid from the phone server doesn't exist yet in the prod server, insert the whole record .

So obviously my syntax is wrong on the insert, just looking for some guidance on how to insert all new records from server1.session to server2.session when this script is run every 15 minutes.

//Defining credentials
$servername = "";
$username = "";
$password = "";

$servername2 = "";
$username2 = "";
$password2 = "";

// Create connection
$conn = new mysqli($servername, $username, $password);
$conn2 = new mysqli($servername2, $username2, $password2);

// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
echo "Connected successfully";

// Check connection2
if ($conn2->connect_error) {
die("Connection failed: " . $conn2->connect_error);
}
echo "Connected successfully";


//Query to select * from Session table on server 1
$query1 = "select * from cdrdb.session";
$results = mysqli_query($conn1, $query1);

foreach ($results as $r => $result) {

}

$stmt1 = mysqli_prepare($conn2, "insert into ambition.session a where not 
exists(a.SESSIONID)");
mysqli_stmt_execute($stmt1) or die(mysqli_error($conn2));

Answer 1

I think the most appropriate way would have to be with a "log" table containing IDs.

However, I imagine the rows IDs are primary keys.

So, you want to do a script where you :

1. Get all rows from the source table
2. Loop on it with a foreach
3. Switch database
4. Do a SQL check on the destination database : SELECT id FROM your_table WHERE id = $id
5. If you have results (check with numRows()), don't do anything.
6. If you don't, then insert in the destination table with the value from foreach

Edit : Basically, you should do something like this :

$source_db = array(
    'host' => 'source_host',
    'name' => 'source_name',
    'user' => 'source_user',
    'pass' => 'source_pass'
);

$dest_db = array(
    'host' => 'dest_host',
    'name' => 'dest_name',
    'user' => 'dest_user',
    'pass' => 'dest_pass'
);

$source_conn = new mysqli($source_db['host'], $source_db['user'], $source_db['pass'], $source_db['name']);
$dest_conn = new mysqli($dest_db['host'], $dest_db['user'], $dest_db['pass'], $dest_db['name']);

// Get all rows from source database
$source_data = $source_conn->query("SELECT * FROM source_table");

// Loop on the results
while($source_item = $source_data->fetch_assoc()) {

    // Check if row exists in dest database
    $row_exists = $dest_conn->query("SELECT id FROM dest_table WHERE id = '".$source_item['id']."' ");

    // If the query returns FALSE, the row does not exist
    if(!$row_exists) {

        // Insert the new row in dest database
        $dest_conn->query("INSERT INTO dest_table (id, name) VALUES ('".$source['id']."', '".$source['name']."' ");
    }

}

Answer 2

In SQL the EXISTS clause (?) is a subquery and looks like this (...WHERE EXISTS(SELECT ... FROM table ...)) Also you didn't state in the insert statement what to insert. So i would do something like this:

INSERT ambition.session a (col_a, col_b, ...) SELECT col_a, col_b, ... FROM cdrdb.session c WHERE NOT EXISTS (SELECT 1 FROM ambition.session a2 WHERE a2.SESSIONID = c.SESSIONID);

Or you can do an INSERT IGNORE ... which will ignore the insertion if it would result in duplicate key and omit the whole WHERE clause, but i don't know if this would bring any performance issue.

I read that you may leave out the column names, but i don't know if it is true. Then:

INSERT ambition.session a SELECT * FROM cdrdb.session c WHERE NOT EXISTS (SELECT 1 FROM ambition.session a2 WHERE a2.SESSIONID = c.SESSIONID);