How to handle chunked encoding request properly?

Question

I have two websites: one with Lighttpd with PHP and second with Apache and neither of this handle chunked transfer encoding properly.

I send this request from my mobile, J2ME and there is no possibility to change this transfer type to any other.

So my only way to go is to handle chunked transfer enconcoded requests in other way. Any solution will be good as long as I can enable it on my CentOS server where I can install and change everything that will be necessary.

So my question is: how to handle chunked encoding request properly on server side?

Answer 1

EDIT: Which version of PHP/Apache/LightHTTP are you running? As there was this bug in PHP before, but it seems to be gone with 5.2.13 and 5.3.2.

If the link above doesn't help, I'd like to know exactly what PHP is seeing, can you put this in your api and post the results? (edited down of course).

$input = file_get_contents('php://input');
$stdin = file_get_contents('php://stdin');

print "FILES: ";
print_r($_FILES);

print("<br>POST: ");
print_r($_POST);

print("<br>input: ".$input);
print("<br>stdin: ".$stdin);
die;

This way we can see what PHP is seeing, and if it's NOT decoding chunked encoding, then we CAN manually decode it.

End EDIT. (leaving the below in case someone else finds it useful)

I assume this is a followup to your previous question. And I'm assuming you're reading the stream from PHP?

I wrote this a few years back, it reads from a chunked encoded stream, you can then do whatever you want with the output. If it's a large file don't read it into a string, instead write to a file.

<?php
define('CRLF', "\r\n");
define('BUFFER_LENGTH', 8192);
$headers = '';
$body = '';
$length = 0;

$fp = fsockopen($host, $port, $errno, $errstr, $timeout);

// get headers FIRST
do
{
    // use fgets() not fread(), fgets stops reading at first newline
    // or buffer which ever one is reached first
    $data = fgets($fp, BUFFER_LENGTH);
    // a sincle CRLF indicates end of headers
    if ($data === false || $data == CRLF || feof($fp)) {
        // break BEFORE OUTPUT
        break;
    }
    $headers .= $data;
}
while (true);
// end of headers

// read from chunked stream
// loop though the stream
do
{
    // NOTE: for chunked encoding to work properly make sure
    // there is NOTHING (besides newlines) before the first hexlength

    // get the line which has the length of this chunk (use fgets here)
    $line = fgets($fp, BUFFER_LENGTH);

    // if it's only a newline this normally means it's read
    // the total amount of data requested minus the newline
    // continue to next loop to make sure we're done
    if ($line == CRLF) {
        continue;
    }

    // the length of the block is sent in hex decode it then loop through
    // that much data get the length
    // NOTE: hexdec() ignores all non hexadecimal chars it finds
    $length = hexdec($line);

    if (!is_int($length)) {
        trigger_error('Most likely not chunked encoding', E_USER_ERROR);
    }

    // zero is sent when at the end of the chunks
    // or the end of the stream or error
    if ($line === false || $length < 1 || feof($fp)) {
        // break out of the streams loop
        break;
    }

    // loop though the chunk
    do
    {
        // read $length amount of data
        // (use fread here)
        $data = fread($fp, $length);

        // remove the amount received from the total length on the next loop
        // it'll attempt to read that much less data
        $length -= strlen($data);

        // PRINT out directly
        #print $data;
        #flush();
        // you could also save it directly to a file here

        // store in string for later use
        $body .= $data;

        // zero or less or end of connection break
        if ($length <= 0 || feof($fp))
        {
            // break out of the chunk loop
            break;
        }
    }
    while (true);
    // end of chunk loop
}
while (true);
// end of stream loop

// $body and $headers should contain your stream data
?>