[英]Break up memory hogging PHP script with nested for loops
I am trying to build a table of part numbers with this script. 我正在尝试使用此脚本构建零件编号表。 I am quickly running out of memory.
我很快用完了内存。 I have commented out the loops I couldn't even get to:
我已经注释掉了我什至无法到达的循环:
foreach ($mat_codes as $mat_code) {
for ($n = 0; $n < count($shape_codes); $n++){
for ($d1 = 0; $d1 <= 12; $d1++){
for ($d1_dec = 1; $d1_dec <= 16; $d1_dec++){
for ($d2 = 0; $d2 <= 12; $d2++){
//for ($d2_dec = 1; $d2_dec <= 16; $d2_dec++){
$build_part = (object) NULL;
$build_part = $mat_code->part_code;
$build_part .= $shape_codes[$n]->part_code;
$build_part .= '.';
if ($d1 != 0) {
$build_part .= $d1 . '-';
}
$build_part .= $d1_dec;
$build_part .= '.';
// if ($d2 != 0) {
// $build_part .= $d2 . '-';
// }
// $build_part .= $d2_dec;
// $build_part .= '.';
// * save $build_part to MYSQL * //
//}
}
}
}
}
}
Unsurprisingly this is returning 毫不奇怪,这正在返回
PHP Fatal error: Allowed memory size of 134217728 byets exhauseted (tried to allocate 71 bytes)
PHP致命错误:耗尽了134217728个字节的允许内存大小(试图分配71个字节)
For the record there are about 16 values in $mat_codes
and 16 in $shape_codes
. 为了记录有大约16个值
$mat_codes
和16 $shape_codes
。 By my calculations, I'm trying to create 692,224 part numbers in this example alone. 根据我的计算,仅在此示例中,我试图创建692,224个部件号。 I think I'm going to add more and have up to ~34 million.
我想我会增加更多,并达到3400万。 IMHO, it would be silly to try and get this done by just allocating more memory to the job.
恕我直言,仅通过为作业分配更多的内存来尝试完成此操作将很愚蠢。 My question -- of course -- how do I break this script up into smaller chunks?
我的问题-当然-如何将这个脚本分成较小的块?
Edit: I have a workaround. 编辑:我有一个解决方法。 I apologize it is out of the scope I defined in the original question.
我很抱歉,这超出了我在原始问题中定义的范围。 I didn't mention that this is simply a table of potential part numbers.
我没有提到这只是潜在零件号的表格。 Particularly, all the potential part numbers allowed by the logic of the part numbering system.
特别是,零件编号系统的逻辑允许的所有潜在零件编号。 Rather than have a master list of part numbers at the ready just waiting to be used I figure I might be better off validating new part input against the part numbering logic, a function like
could_this_be_a_part_number($input)
. 我认为,与其在等待使用时准备好零件号的主列表,不如根据零件编号逻辑来验证新的零件输入,可能会更好,像
could_this_be_a_part_number($input)
这样的函数。 It could just test the inputted string against the part numbering logic as opposed to having some master table I would need to check against. 它可以根据零件编号逻辑测试输入的字符串,而不是拥有一些我需要检查的主表。
Edit x2: I'm sorry this is the worst. 编辑x2:对不起,这是最糟糕的情况。 I moved
unset($build_part)
into the innermost for loop. 我将
unset($build_part)
移到了最里面的for循环中。 The problem was I was trying to a development function in Drupal to print out each $build_part before it was unset - dpm($build_part)
. 问题是我试图在Drupal中使用开发功能在未设置$ build_part之前将其打印出来
dpm($build_part)
。 The script runs fine, its trying to build a webpage with that many part numbers printed to it is what breaks the script. 该脚本运行良好,它试图构建一个带有许多部件号的网页,这是破坏脚本的原因。
First of all: $build_part seems like it's not supposed to be an object, because you treat it like a string, so don't initialize it like that. 首先: $ build_part似乎不应该是一个对象,因为您将其视为字符串,所以不要这样初始化它。
$build_part = NULL;
will suffice. 就足够了。
Second of all: We don't know how you'd like your data to be saved in your database, but you could (as my original answer somehow proposed) build up a query, send it to the database and then continue. 第二:我们不知道您希望如何将数据保存在数据库中,但是您可以(按照我的原始答案提出)建立一个查询,将其发送到数据库,然后继续。
Now let's have a look at your code: 现在让我们看一下您的代码:
$i = 0;
foreach ($mat_codes as $mat_code) {
for ($n = 0; $n < count($shape_codes); $n++){
for ($d1 = 0; $d1 <= 12; $d1++){
for ($d1_dec = 1; $d1_dec <= 16; $d1_dec++){
for ($d2 = 0; $d2 <= 12; $d2++){
for ($d2_dec = 1; $d2_dec <= 16; $d2_dec++){
$i++;
$build_part = (object) NULL;
$build_part = $mat_code->part_code.".";
$build_part .= $shape_codes[$n]->part_code;
$build_part .= '.';
if ($d1 != 0) {
$build_part .= $d1 . '-';
}
$build_part .= $d1_dec;
$build_part .= '.';
if ($d2 != 0) {
$build_part .= $d2 . '-';
}
$build_part .= $d2_dec;
// echo "$i $build_part <br>";
// * save $build_part to MYSQL * //
}
// This is first triggered when the count reaches 16 items
echo "--- lvl 5<br>";
}
// 208 items
echo "--- lvl 4<br>";
}
// 3328 items
echo "--- lvl 3<br>";
}
echo "--- lvl 2<br>";
}
echo "--- lvl 1<br>";
}
I have determined (and you can do it somewhere else) the level 3 sign as the point where you could be sending some pre-constructed MySQL query to the database. 我已经确定(您可以在其他地方使用) 3级标志作为向数据库发送一些预先构造的MySQL查询的起点。 Something like:
就像是:
$mysqli->query("INSERT INTO parts (
part_id ) VALUES $very_long_string;");
$mysqli->query("INSERT INTO parts (
part_id ) VALUES $very_long_string;");
Also look at this MySQL command: INSERT DELAYED , I think maybe it might (sic!) be quite useful in your situation. 还要查看以下MySQL命令: INSERT DELAYED ,我认为在您所处的情况下它可能很有用。
$very_long_string would be built in the most inner loop like: $ very_long_string将构建在最内部的循环中,例如:
$very_long_string .= "('$build_part'),";
and in the end (before we create and send the final query) we would cut the last comma: 最后(在创建并发送最终查询之前),我们将剪切最后一个逗号:
$very_long_string = rtrim($very_long_string, ",");
Then send the 3000+ items long query into the database. 然后将3000多个项目的长查询发送到数据库中。 Then unset($very_long_string) and start again.
然后取消设置($ very_long_string)并重新开始。
So I guess the final code would look something like this: 所以我想最终的代码看起来像这样:
foreach ($mat_codes as $mat_code) {
for ($n = 0; $n < count($shape_codes); $n++){
for ($d1 = 0; $d1 <= 12; $d1++){
$very_long_string = null;
for ($d1_dec = 1; $d1_dec <= 16; $d1_dec++){
for ($d2 = 0; $d2 <= 12; $d2++){
for ($d2_dec = 1; $d2_dec <= 16; $d2_dec++){
$build_part = NULL;
$build_part = $mat_code->part_code.".";
$build_part .= $shape_codes[$n]->part_code;
$build_part .= '.';
if ($d1 != 0) {
$build_part .= $d1 . '-';
}
$build_part .= $d1_dec;
$build_part .= '.';
if ($d2 != 0) {
$build_part .= $d2 . '-';
}
$build_part .= $d2_dec;
$very_long_string .= "('$build_part'),";
}
}
}
$very_long_string = rtrim($very_long_string, ",");
$mysqli->query("INSERT INTO parts (part_id) VALUES $very_long_string;");
unset($very_long_string);
}
}
}
EDIT after reading your EDIT: I was of course dealing with the fact that you wanted to populate the database :D .. If you need a validating function, then that's a whole another story. 阅读您的EDIT后进行的EDIT:我当然正在处理您想填充数据库的事实:D ..如果您需要验证函数,那完全是另一回事了。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.