How do I get the body from this example HTTP request?

Question

I'm trying to find the simplest way to parse an RFC-822 document in Java. Assume that I have a message-queue on which HTTP messages are stored. Both requests and responses. So they are not retrieved in the "normal" way by making a socket-connection to - say - port 80 and sending/retrieving the message from there.

In the code below, I deliberately mixed "mail" headers with a HTTP message. It's meant as a demonstration that the two are not very different. But that's beside the point. Here's the code:

package httpexample;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import org.apache.http.Header;
import org.apache.http.HttpException;
import org.apache.http.HttpRequest;
import org.apache.http.impl.io.DefaultHttpRequestParser;
import org.apache.http.impl.io.HttpTransportMetricsImpl;
import org.apache.http.impl.io.SessionInputBufferImpl;
import org.apache.http.io.HttpMessageParser;
import org.apache.http.message.BasicHttpEntityEnclosingRequest;

public class HttpExample {

    // RFC 822

    public static void main(String[] args) throws IOException, HttpException {
        String str = "POST http://localhost:8080/foobar/1234567 HTTP/1.1\n" +
            "Message-ID: <19815303.1075861029555.JavaMail.ss@kk>\n" +
            "Date: Wed, 6 Mar 2010 12:32:20 -0800 (PST)\n" +
            "From: someone@someotherplace.com\n" +
            "To: someone@someplace.com\n" +
            "Subject: some subject\n" +
            "Mime-Version: 1.0\n" +
            "Content-Type: text/plain; charset=us-ascii\n" +
            "Content-Transfer-Encoding: 7bit\n" +
            "X-From: one, some <some.one@someotherplace.com>\n" +
            "X-To: one\n" +
            "X-cc: \n" +
            "X-bcc: \n" +
            "X-Origin: Bob-R\n" +
            "X-FileName: rbob (Non-Privileged).pst\n" +
            "\n" +
            "some message\n";
        ByteArrayInputStream fakeStream = new ByteArrayInputStream(
                str.getBytes());
        HttpTransportMetricsImpl metrics = new HttpTransportMetricsImpl();
        SessionInputBufferImpl inbuffer = new SessionInputBufferImpl(metrics, 1024);

        inbuffer.bind(fakeStream);
        HttpMessageParser<HttpRequest> requestParser =
                new DefaultHttpRequestParser(inbuffer);
        BasicHttpEntityEnclosingRequest request = (BasicHttpEntityEnclosingRequest)requestParser.parse();

        for (Header hdr : request.getAllHeaders()) {
            System.out.println(String.format("%-30s = %s", hdr.getName(), hdr.getValue()));
        }
        System.out.println(String.format("Request Line: %s", request.getRequestLine()));
        System.out.println(String.format("Body\n------------------\n%s",
                request.getEntity()));
    }

}

The output looks like this:

Message-ID                     = <19815303.1075861029555.JavaMail.ss@kk>
Date                           = Wed, 6 Mar 2010 12:32:20 -0800 (PST)
From                           = someone@someotherplace.com
To                             = someone@someplace.com
Subject                        = some subject
Mime-Version                   = 1.0
Content-Type                   = text/plain; charset=us-ascii
Content-Transfer-Encoding      = 7bit
X-From                         = one, some <some.one@someotherplace.com>
X-To                           = one
X-cc                           = 
X-bcc                          = 
X-Origin                       = Bob-R
X-FileName                     = rbob (Non-Privileged).pst
Request Line: POST http://localhost:8080/foobar/1234567 HTTP/1.1
Body
------------------
null

What I can't figure out, is how to access the body of the message.

I would expect it to have the content some message\\n

I can't find any method in BasicHttpEntityEnclosingRequest that would give me this value. In an earlier version I used

HttpRequest request = requestParser.parse();

instead of

BasicHttpEntityEnclosingRequest request = 
    (BasicHttpEntityEnclosingRequest) requestParser.parse();

I changed it to BasicHttpEntityEnclosingRequest because that has the getEntity method. But that returns null .

So I'm a bit lost.

Where do I find the body?

Answer 1

I have added Content-Length header, otherwise the parser simply ignores the POST body. I have modified your code, now it parses the body just fine:

package org.apache.http.examples;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;

import org.apache.http.Header;
import org.apache.http.HttpException;
import org.apache.http.message.BasicHttpEntityEnclosingRequest;
import org.apache.http.util.EntityUtils;

public class HttpExample {

    // RFC 822

    public static void main(String[] args) throws IOException, HttpException {
        String str = "POST http://localhost:8080/foobar/1234567 HTTP/1.1\n" +
            "Message-ID: <19815303.1075861029555.JavaMail.ss@kk>\n" +
            "Date: Wed, 6 Mar 2010 12:32:20 -0800 (PST)\n" +
            "From: someone@someotherplace.com\n" +
            "To: someone@someplace.com\n" +
            "Subject: some subject\n" +
            "Mime-Version: 1.0\n" +
            "Content-Type: text/plain; charset=us-ascii\n" +
            "Content-Transfer-Encoding: 7bit\n" +
            "X-From: one, some <some.one@someotherplace.com>\n" +
            "X-To: one\n" +
            "X-cc: \n" +
            "X-bcc: \n" +
            "X-Origin: Bob-R\n" +
            "X-FileName: rbob (Non-Privileged).pst\n" +
            "Content-Length: 13\n" +
            "\n" +
            "some message\n";
        ByteArrayInputStream fakeStream = new ByteArrayInputStream(
                str.getBytes());

        BHttpConnectionBaseImpl b = new BHttpConnectionBaseImpl(fakeStream);

        BasicHttpEntityEnclosingRequest request1 = (BasicHttpEntityEnclosingRequest) b.receiveRequestHeader();
        b.receiveRequestEntity(request1);


        for (Header hdr : request1.getAllHeaders()) {
            System.out.println(String.format("%-30s = %s", hdr.getName(), hdr.getValue()));
        }
        System.out.println(String.format("Request Line: %s", request1.getRequestLine()));
        System.out.println(String.format("Body\n------------------\n%s",
                EntityUtils.toString( request1.getEntity() ) ));
    }

}

class BHttpConnectionBaseImpl extends  org.apache.http.impl.DefaultBHttpServerConnection{

    private InputStream inputStream;

    public BHttpConnectionBaseImpl(final InputStream inputStream) {
        super(4048);
        this.inputStream = inputStream;
        try {
            super.bind(new Socket());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    @Override
    protected InputStream getSocketInputStream(final Socket socket) throws IOException {
        return inputStream;
    }

    @Override
    protected OutputStream getSocketOutputStream(final Socket socket) throws IOException {
        return new ByteArrayOutputStream();
    }
}

The parsing of POST body happens in org.apache.http.impl.BHttpConnectionBase.prepareInput(HttpMessage) , whoever its only constructor is protected and requires a lot of parameters. The child org.apache.http.impl.DefaultBHttpServerConnection has a convenient public constructor and does the header parsing in receiveRequestHeader() . The methods I'm overloading are need to bypass some error checks, eg if the Socket == null and to be able to read the request from the fakeStream

Another approach that might working, although I have not tested it, is to override Socket particularly its getInputStream() and getOutputStream() . Then create an instance of DefaultBHttpServerConnection and call its bind method. The rest should be the same.

Answer 2

I think the problem might be that it's not clear from your message headers what's the length of the body, so the receiver just ignores it. HTTP specification defines several options on how to convey this information and none of them seems to be applied here:

1. Content-Transfer-Encoding must be Transfer-Encoding
2. 7bit is not among the standard options .
3. When you use str.getBytes() , it gives you UTF-16 bytes which is not us-ascii declared in Content-Type .

So, I would change your request a bit:

1. Use header Content-Type: text/plain; charset=UTF-16 Content-Type: text/plain; charset=UTF-16
2. Remove header Content-Transfer-Encoding
3. Add Content-Lenght: 28 (28 being "some message\\n".getBytes().length() ).

Answer 3

Looking at the source code of DefaultHttpRequestParser it seems it only parses the request line and the headers, it does not attempt to parse the body.

This thread is discussing the same topic. There are few solution proposals as well.

Answer 4

custom parse head through overwirte the LineParser:

inbuffer = new SessionInputBufferImpl(new HttpTransportMetricsImpl(), reqDataLength);
inbuffer.bind(input);
HttpMessageParser<org.apache.http.HttpRequest> requestParser = new DefaultHttpRequestParser(
                inbuffer,
                new LineParser(),
                new DefaultHttpRequestFactory(),
                MessageConstraints.DEFAULT
        );

get the entity body as follows:

        HttpEntityEnclosingRequest ereq = (HttpEntityEnclosingRequest) req;
        ContentLengthStrategy contentLengthStrategy =
                    StrictContentLengthStrategy.INSTANCE;
        long len = contentLengthStrategy.determineLength(req);
        InputStream contentStream = null;
        if (len == ContentLengthStrategy.CHUNKED) {
            contentStream = new ChunkedInputStream(buf);
        } else if (len == ContentLengthStrategy.IDENTITY) {
            contentStream = new IdentityInputStream(buf);
        } else {
            contentStream = new ContentLengthInputStream(buf, len);
        }
        BasicHttpEntity ent = new BasicHttpEntity();
        ent.setContent(contentStream);
        ereq.setEntity(ent);
        return ereq;