使用Future在App引擎上异步urlfetch Http发布

Question

My goal is to rapidly make posts to a server from appengine(java). I am attempting to do this using UrlFetchService.fetchAsync. I have been basing my code after this blog post . I have been able to make the request using the code below, however I get some strange behavior:

private void futureRequests() {
    URLFetchService fetcher = URLFetchServiceFactory.getURLFetchService();
    URL url = new URL("https://someserver.com");

    FetchOptions fetchOptions = FetchOptions.Builder.withDefaults();
    fetchOptions.doNotValidateCertificate();
    fetchOptions.setDeadline(60D);

    ArrayList<Future<HTTPResponse>> asyncResponses = new ArrayList<Future<HTTPResponse>>();

    for (int i = 0; i < postDatas.size(); i++) {

        HTTPRequest request = new HTTPRequest(url, HTTPMethod.POST, fetchOptions);
        request.setPayload(postDatas.get(i).getBytes(UTF8));
        HTTPHeader header = new HTTPHeader("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
        request.setHeader(header);
        header = new HTTPHeader("Content-Length", Integer.toString(postDatas.get(i).getBytes().length));
        request.setHeader(header);
        header = new HTTPHeader("Authorization", "auth=" + authToken);
        request.setHeader(header);
        Future<HTTPResponse> responseFuture = fetcher.fetchAsync(request);
        asyncResponses.add(responseFuture);
    }

    for (Future<HTTPResponse> future : asyncResponses) {
        HTTPResponse response;
        try {
            response = future.get();
            int responseCode = response.getResponseCode();
            resp.getWriter().println("response: " + responseCode);
            logger.warning("Response: " + responseCode);

        } catch (Exception e) {
        }
    }
}

The strange behavior is that I get duplicate posts on the server, and according to my appstats page I use 10x-20x more urlFetches than what was added with the code above. Below is my appstats screen:

There are more urlFetch calls that could not fit on the screen. It appears that the requests are still completing in a synchronous fashion(circled items), but there are many urlFetches that appear to go on at the same time. My question is how am I getting all this calls to urlFetch when I only had 14 Future ?? Could the server be giving an error or 503 and urlFetch retrying until it goes through? And how can I be getting 2 posts for each request??

I understand that I could use the task queue to do asyc request, however I am dealing with a relatively low number of request(20-100) and the cold start time of ramping up another instance would probably make this not a good option for my situation. Can anyone explain this behavior or have experience with this?

Answer 1

这仅仅是我的代码中的一个错误，导致我的应用程序发出了比我想象的更多的请求。