[英]php - how to handle cURL sending large size json data in POST efficiently?
Here is the scenario. 这是场景。 There is a method which fetches 156000 rows from DB table.
有一种方法可以从数据库表中提取156000行。 I need to send this data to another
php
file which uses phpexcel
to convert it into Excel file. 我需要将此数据发送到另一个使用
phpexcel
将其转换为Excel文件的php
文件中。
Below is the code of curl
post request. 以下是
curl
请求后的代码。
if($_SERVER['SERVER_NAME'] === 'localhost'){
$domain = $_SERVER['REMOTE_ADDR'].':'.$_SERVER['SERVER_PORT'];
}else{
$domain = $_SERVER['SERVER_NAME'].':'.$_SERVER['SERVER_PORT'];
}
$prefix = $_SERVER['HTTPS'] ? 'https://' : 'http://';
$relative = '/rest/ExportExcelCsv.php';
error_log($prefix.$domain.$relative);
// converting array of returned rows into json
$postData = json_encode($utilDataArray['tableData']);
$defaults = array(
CURLOPT_URL => $prefix.$domain.$relative,
CURLOPT_HEADER => 0,
CURLOPT_RETURNTRANSFER => TRUE,
CURLOPT_TIMEOUT => 500,
CURLOPT_PROXY => "<proxy_url>",
CURLOPT_PROXYPORT => 80,
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => array('data' => $postData)
);
$ch = curl_init();
curl_setopt_array($ch, ($defaults));
// curl_setopt($ch, CURLOPT_HTTPHEADER, array('Accept: application/json'));
// curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
$result = curl_exec($ch);
if( !$result )
{
trigger_error(curl_error($ch));
}
var_dump($result);
curl_close($ch);
In a statement, CURLOPT_POSTFIELDS => array('data' => $postData)
, I am passing postData in an array with key data as ExportExcelCsv.php gets post data in that format with json_decode($_POST['data'], true)
. 在声明中,
CURLOPT_POSTFIELDS => array('data' => $postData)
,我正在通过具有关键数据的数组传递postData,因为ExportExcelCsv.php使用json_decode($_POST['data'], true)
来获取该格式的发布数据json_decode($_POST['data'], true)
。
First issue is the size of "POST Content-Length". 第一个问题是“ POST内容长度”的大小。 So, I've increased it with a statement
ini_set('post_max_size', '100M'); ini_set('upload_max_filesize', '100M');
因此,我使用一条语句
ini_set('post_max_size', '100M'); ini_set('upload_max_filesize', '100M');
增加了它ini_set('post_max_size', '100M'); ini_set('upload_max_filesize', '100M');
ini_set('post_max_size', '100M'); ini_set('upload_max_filesize', '100M');
in both php files. 在两个PHP文件。 Still it didn't work.
仍然没有用。 The size of post content is almost 42MB.
帖子内容的大小几乎为42MB。
How do I send such large size json data in curl post request ? 如何在curl post请求中发送如此大的json数据? And how do to handle it efficiently ?
以及如何有效地处理它? Is there anyway to send data in chunk ?
反正有块发送数据吗? If yes, then how to handle that data while generating excel file out of it ?
如果是,那么如何在从中生成excel文件的同时处理该数据?
First off, post_max_size
and upload_max_filesize
need to be in .htaccess, a vhost or in php.ini, not via ini_set
. 首先,
post_max_size
和upload_max_filesize
必须位于.htaccess,虚拟主机或php.ini中,而不是通过ini_set
。 These, along with a few other ini values MUST be set before your script starts. 这些,以及一些其他ini值必须在脚本启动之前设置。
You can also probably also make the process a bit easier by first dumping your json data into a text file and sending the file via curl, rather than sending the data directly. 您还可以通过首先将json数据转储到文本文件中并通过curl发送文件,而不是直接发送数据,来简化此过程。 A well written answer on how to send a file via curl can be found here: https://stackoverflow.com/a/15200804/4027341
在此处可以找到有关如何通过curl发送文件的书面答复: https : //stackoverflow.com/a/15200804/4027341
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.