[英]PHP/MySQL - Multiple queries at the same time
I have 24 databases with a table labeled email_queue
.我有 24 个数据库,其中有一个标有
email_queue
的表。
I have another database with a list of all the databases that have the email_queue
table in it.我有另一个数据库,其中列出了所有包含
email_queue
表的数据库。
I loop through the list of databases and query my email_queue
table to send mails for each database.我遍历数据库列表并查询我的
email_queue
表以发送每个数据库的邮件。
The problem with this is that the php script gets held up on, lets say, the 3rd database while sending 500 emails leaving the other databases after that one to wait for their turn.这样做的问题是 php 脚本在发送 500 封电子邮件时被阻止在第三个数据库上,然后离开其他数据库等待轮到他们。
I am trying to figure out how i can query all 24 databases at the same time and send the email queue at the same time.我想弄清楚如何同时查询所有 24 个数据库并同时发送电子邮件队列。
Any suggestions?有什么建议?
I think having this many databases is probably a sign of bad design.我认为拥有这么多数据库可能是设计不佳的标志。 If you can't change it and need to move forward now, I suggest one of two options:
如果你不能改变它并且现在需要继续前进,我建议两种选择之一:
Here's a somewhat complete example using the mysqli extension (requires the mysqlnd driver):这是一个使用 mysqli 扩展的比较完整的例子(需要 mysqlnd 驱动程序):
$credentials = array(
array(
'host' => 'host1',
'user' => 'user',
'password' => 'password',
'database' => 'database'
),
array(
'host' => 'host2',
'user' => 'user',
'password' => 'password',
'database' => 'database'
),
// credentials for other sites
);
$dbcs = array();
foreach ($credentials as $config) {
$dbcs[] = array($db = new mysqli(
$config['host'],
$config['user'],
$config['pass'],
$config['database']
));
$query = ""; // here is your query to do whatever it is with your table
$db->query($query, MYSQLI_ASYNC);
}
$results = array();
$errors = array();
$rejected = array();
$secondsToWait = 1;
while (!empty($dbcs)) {
foreach ($dbcs as $key => $c) {
$db = $c[0];
if (mysqli_poll($c, $errors, $rejected, $secondsToWait) == 1) {
$r = $db->reap_async_query();
// here you would do your fetches for each query, such as
$results[] = $r->fetch_assoc();
// do what you need to do with the result
// then cleanup
$r->free();
$db->close();
unset($dbcs[$key]);
}
}
}
Note that it does have drawbacks, such as a failed query may bring down the whole program.请注意,它确实有缺点,例如失败的查询可能会导致整个程序崩溃。
On way to do this is with curl_multi_open这样做的方法是使用 curl_multi_open
Split your script into two, you can make one php file (say email_out.php) take the db name (or some variable that's used to look up the db name, either the switch will be in the for loop or in email_out.php), and then do the mass email based of that one script.将你的脚本分成两部分,你可以让一个 php 文件(比如 email_out.php)使用 db 名称(或者一些用于查找 db 名称的变量,开关将在 for 循环中或在 email_out.php 中) ,然后根据该脚本进行群发电子邮件。
the second part uses curl_multi_open to open the email_out.php script multiple times, effectively creating multiple separate connections to different db's, the scripts can all resolve at different times since they are all running in parallel.第二部分使用 curl_multi_open 多次打开 email_out.php 脚本,有效地创建到不同数据库的多个单独连接,脚本可以在不同时间解析,因为它们都是并行运行的。 Essentially, your loop is now adding the script to curl_multi_open multiple times with different arguments and then executing all of them asynchronously.
本质上,您的循环现在使用不同的参数多次将脚本添加到 curl_multi_open,然后异步执行所有这些。
class Fork
{
private $_handles = array();
private $_mh = array();
function __construct()
{
$this->_mh = curl_multi_init();
}
function add($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_multi_add_handle($this->_mh, $ch);
$this->_handles[] = $ch;
return $this;
}
function run()
{
$running=null;
do {
curl_multi_exec($this->_mh, $running);
usleep (250000);
} while ($running > 0);
for($i=0; $i < count($this->_handles); $i++) {
$out = curl_multi_getcontent($this->_handles[$i]);
$data[$i] = json_decode($out);
curl_multi_remove_handle($this->_mh, $this->_handles[$i]);
}
curl_multi_close($this->_mh);
return $data;
}
}
(from http://gonzalo123.com/2010/10/11/speed-up-php-scripts-with-asynchronous-database-queries/ ) (来自http://gonzalo123.com/2010/10/11/speed-up-php-scripts-with-asynchronous-database-queries/ )
So your loop would look something like this:所以你的循环看起来像这样:
$fork = new Fork;
for ($i = 0; $i < 24; $i++) {
$fork->add("email_out.php?t=" . $i);
}
$fork->run();
In your script try this.在你的脚本中试试这个。
set_time_limit(0);
set_time_limit(0);
in order to override PHP's max_execution_timegetopt()
function to get the database name when the script is ran from the command line (ie php script.php -d database1
).getopt()
函数获取数据库名称(即php script.php -d database1
)。-d
) I specified in 2. If you have 20 databases then you must have 20 entries.-d
) 发送电子邮件的每个数据库创建一个条目。如果您有 20 个数据库,那么您必须有 20 个条目。 Using this way you will see a separate PHP process for each cron job and you can isolate a database if ever you will encounter an error with it.使用这种方式,您将看到每个 cron 作业都有一个单独的 PHP 进程,如果遇到错误,您可以隔离数据库。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.