How to handle 419 http response in Java

Question

I am trying to parse through a particular website and am getting an HTTP response code of 419 when my java code calls it. I need to parse through the response to find content and I am stuck on the response code.

I have tried putting together a Java program using apache http client(version 4.5.6) to call a website that I need to parse. The http response code I get back is 419.

try (CloseableHttpClient httpclient = HttpClients.createDefault()) {
    HttpGet httpGet = new HttpGet("http://www.website.com");
    try (CloseableHttpResponse response1 = httpclient.execute(httpGet)) {
        System.out.println(response1.getStatusLine());
        HttpEntity entity1 = response1.getEntity();
        EntityUtils.consume(entity1);
    }
}

The result that it prints out is this:

HTTP/1.1 419 status code 419

I am expecting a 200

HTTP/1.1 200 OK

I get that when I change the website to google or other sites.

Answer 1

I was making a get request through HttpClient library as well as from POSTMAN and facing same 419 error. To solve this 419 error we need to add csrf token while making form submission.

However, In-case if you are still wondering how to find csrf token even when you are making a GET request and facing status 419 . In my case I solved the problem by adding the user-agent: xxxx token in header .

Example:

user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36

HttpClient Code:

  connectionManager = new PoolingHttpClientConnectionManager();
  ...
  ...
  ...

  httpClient = HttpClients.custom()
                .setConnectionManager(connectionManager)
                .setRedirectStrategy(new LaxRedirectStrategy())
                .setUserAgent("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36")
                .build();