Java Apache http library file upload issue

Question

I Have a webservice that uploads a file. (details in the first link)

To communicate with this webservice I use org.apache.http library (Apache HttpComponents). I have found most of this code if not all of it here .

Sadly the solution only works with images and despite the efforts when trying to upload a video is shows the error of Content Length too long . To try and fix this I decided to replace it with what's in current use. At first using HttpClientBuilder

HttpClient httpclient = HttpClientBuilder.create().disableContentCompression().build();

My try failed as it still gives the same error and after research I found out that I need to use the CloseableHttpClient as follow to disable the automatic headers

CloseableHttpClient httpclient = HttpClients.custom()
        .setHttpProcessor(HttpProcessorBuilder.create().build())
        .build();

That worked but when setting the headers something doesn't pass I can't figure it out. I did it as follow

httppost.addHeader("Keep-Alive", "timeout=5, max=100");
httppost.addHeader("Connection", "Keep-Alive");
httppost.addHeader("Content-Type", "text/html; charset=UTF-8");

But when tracing the response this is what I have

Http Client Builder Trace
    Date,Tue, 28 Mar 2017 18:30:50 GMT
    Server,Apache/2.4.23 (Win64) PHP/7.0.10
    X-Powered-By,PHP/7.0.10
    Content-Length,43
    Keep-Alive,timeout=5, max=100
    Connection,Keep-Alive
    Content-Type,text/html; charset=UTF-8


Closable Response Trace
    Date,Tue, 28 Mar 2017 18:31:27 GMT
    Server,Apache/2.4.23 (Win64) PHP/7.0.10
    Content-Length,311
    Connection,close
    Content-Type,text/html; charset=iso-8859-1

I tried to set the headers to the response but I am haven't figured how to instantiate it properly to set the headers. And also I am trying to understand why the httppost is not being taken in consideration. Did I miss something?

One last thing when I use Closable The way I did it doesn't upload anymore due to missing headers I guess.

EDIT

Error when uploading video

Exception in thread "main" org.apache.http.ContentTooLongException: Content length is too long: 16363535
at org.apache.http.entity.mime.MultipartFormEntity.getContent(MultipartFormEntity.java:103)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:199)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:306)
at testfileupload.PostFile.main(PostFile.java:100)
C:\Users\User\AppData\Local\NetBeans\Cache\8.2\executor-snippets\run.xml:53: Java returned: 1
BUILD FAILED (total time: 0 seconds)

EDIT

I changed the max file size for php following this but it still shows the same error

Update

This is how I handle the multipart (I use multipartform)

HttpEntity resEntity = MultipartEntityBuilder.create()
    .addBinaryBody("userfile", file, ContentType.create("video/mp4"), file.getName())
    .build();

I rememeber reading something about it (not sure) And as for the php.ini params that need to be of higher value upload_max_filesize = 40M and post_max_size = 40M and memory_limit = 128M which I am kinda sure its enough (using wamp64).

I kept trying to udnerstand the content length issue since ..

I found out that the HttpEntity sets calculates the length and I checked the value its 2 bits off I believe but no matter the conenction type it just doesn't work and sticks to content length error.

I also tried

HttpClient httpclient = HttpClients.custom().setDefaultHeaders(Arrays.asList(CustomHeader)).build();

But still nothing. At this point I highly doubt its about setting the header and more related to HttpEntity with the Multipart or the HttpPost but It could be else.

Upload

I tried with a 9s Video of a size ~350Kb but it still gives the content length error which means there might be a default content set or none at all

Answer 1

So I dont know what I did / changed but now its working with the error.

I would really want to know why this cause an error it doesn't make sense so far unless its calling something that can't take too high of a value (method of param type given value more than it can)

To be clear the error is generated when I use this (verifcation area)

System.out.println(response.getStatusLine());
if (resEntity != null) {
// error from below, kinda forgot about it
      System.out.println(EntityUtils.toString(resEntity));
}

    if (resEntity != null) {
//      resEntity.consumeContent();
    }

The webservice is the same as in the first link.

Below is my code (dont mind the comments I didn't clear the decrepated / changed / replaced / test / different approaches)

import java.io.File;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.apache.http.Header;
import org.apache.http.HttpEntity;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpResponse;
import org.apache.http.HttpVersion;
import org.apache.http.ProtocolVersion;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.mime.MultipartEntity;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.ContentBody;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.message.BasicHeader;
import org.apache.http.params.CoreProtocolPNames;
import org.apache.http.protocol.HTTP;
import org.apache.http.protocol.HttpProcessorBuilder;
import org.apache.http.util.EntityUtils;


public class PostFile {
  public static void main(String[] args) throws Exception {
//    HttpClient httpclient = new DefaultHttpClient();

//    CloseableHttpClient httpclient = HttpClients.custom()
//        .setHttpProcessor(HttpProcessorBuilder.create().build())
//        .build();

//    Header headerCustom = new BasicHeader(HttpHeaders.CONTENT_LENGTH, "15");


//    HttpClient httpclient = HttpClients.custom().build();

//        HttpClient httpclient = HttpClientBuilder.create().disableContentCompression().build();
   HttpClient httpclient = HttpClientBuilder.create().build();     
//    httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);


    HttpPost httppost = new HttpPost("http://localhost/userVids/upload.php");

    httppost.setProtocolVersion(new ProtocolVersion("HTTP", 1, 1));
    File file = new File("C:/Users/User/Desktop/test_video.mp4");

//    httppost.addHeader("Content-Length", String.valueOf(file.length()));

//    MultipartEntity mpEntity = new MultipartEntity();
//    ContentBody cbFile = new FileBody(file, ContentType.MULTIPART_FORM_DATA);
//    mpEntity.addPart("userfile", cbFile);
//      System.out.println(mpEntity.getContentLength());
//
//    httppost.setEntity(mpEntity);

    /*
        Date,Tue, 28 Mar 2017 18:30:50 GMT
        Server,Apache/2.4.23 (Win64) PHP/7.0.10
        X-Powered-By,PHP/7.0.10
        Content-Length,43
        Keep-Alive,timeout=5, max=100
        Connection,Keep-Alive
        Content-Type,text/html; charset=UTF-8
    */

    /*
        Date,Tue, 28 Mar 2017 18:31:27 GMT
        Server,Apache/2.4.23 (Win64) PHP/7.0.10
        Content-Length,311
        Connection,close
        Content-Type,text/html; charset=iso-8859-1
    */

    HttpEntity resEntity = MultipartEntityBuilder.create()
        .addBinaryBody("userfile", file, ContentType.create("video/mp4"), file.getName())
        .build();

      System.out.println("Entity Length " + resEntity.getContentLength());
//    httppost.addHeader("Keep-Alive", "timeout=5, max=100");
//    httppost.addHeader("Connection", "Keep-Alive");
//    httppost.addHeader("Content-Type", "text/html; charset=UTF-8");

//    httppost.addHeader("Content-Disposition", "form-data; name=\"userfile\"; filename=\"star_empty.png\"");
//    httppost.addHeader("Content-Type", "image/png");
//    httppost.addHeader("Content-Transfer-Encoding", "binary");
//    httppost.addHeader("Content-Length", "99999");

    httppost.setEntity(resEntity);

    System.out.println("executing request " + httppost.getRequestLine());
    HttpResponse response = httpclient.execute(httppost);
//    HttpEntity resEntity = response.getEntity();

    System.out.println("Headers (name,value)");
    List<Header> httpHeaders = Arrays.asList(response.getAllHeaders());        
    httpHeaders.forEach((header) -> {
        System.out.println(header.getName() + "," + header.getValue());
    });

    System.out.println(response.getStatusLine());
    if (resEntity != null) {
      System.out.println(EntityUtils.toString(resEntity));
    }
    if (resEntity != null) {
//      resEntity.consumeContent();
    }

//    httpclient.getConnectionManager().shutdown();
  }
}

BUT A HUGE NOTICE I had my php.ini (changed all of them in wamp didn't know if it would consider only the running version or reads all (didn't test) ) and changed the values as Lucas Holt (he fixed it ie, I am just providing for who needs or maybe explanation) said and he deserves credit for it. (I am sure it didn't work before that I just tested a lot and dont remember really.)