How can I suspend appstats for a single request on App Engine/Java?

Question

I normally run appstats fulltime on my sandbox appid. However, I have one complicated operation (basically rebuilding the stock database) that causes appstats to blow up my instance, throwing OutOfMemoryErrors. Even with larger instance sizes, it still fails. Appstats just wants too much RAM.

I don't need appstats on this request. Ideally I will call a method on whatever ThreadLocal object is responsible for appstats collection and tell it to twiddle its thumbs for a few minutes.

I've considered extending the AppstatsFilter to ignore certain URLs, but the offending request executes as a deferred task and identifying it by path is somewhat complicated.

How can I tell appstats to pause?

Just in case it isn't clear: Uploading a version of my app with appstats disabled, running my task, then uploading a version with appstats enabled is what I'm doing now. I don't want to do this.

Answer 1

What I did is write my own CustomAppstatsFilter and exclude certain url's.

public class CustomAppstatsFilter extends AppstatsFilter {

  @Override
  public void doFilter(ServletRequest request, ServletResponse response,
        FilterChain chain) throws IOException, ServletException {

    if (request instanceof HttpServletRequest) {
        String url = ((HttpServletRequest) request).getRequestURL().toString();

        // We do not want these to show up in appstats
        if ((url.contains("appstats"))
                || url.contains("_ah/")
                || url.contains("cron/")
                || url.contains("batch/")) {
            chain.doFilter(request, response);
            return;
        } else {
            super.doFilter(request, response, chain);
        }
    }
  }
}

EDIT - This can be combined with ZiglioNZ great answer.

<!-- Appstats filter for profiling application -->
<filter>
    <filter-name>appstats</filter-name>
    <filter-class>my.company.filter.CustomAppstatsFilter</filter-class>
    <init-param>
        <param-name>maxLinesOfStackTrace</param-name>
        <param-value>5</param-value>
    </init-param>
</filter>
<filter-mapping>
    <!-- excludes are in CustomAppstatsFilter -->
    <filter-name>appstats</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

Answer 2

Good question. For Python, the answer is simple:

from google.appengine.ext.appstats import recording

class ...(webapp.RequestHandler):
  def get(self):
    recording.dont_record()
    ...

Maybe there's a similar API in Java?

Alternatively, again the Python version has a flexible way of filtering out which requests to record; I think in the Java version you can accomplish a similar thing by using the and entries in your web.xml. (See the Java appstats docs.)

Answer 3

Jeff, I wonder whether this can help you reducing the occurence of OutOfMemoryErrors: How to reduce the memory usage of Appstats on Google App Engine Java

<filter>
  <filter-name>appstats</filter-name>
    <filter-class>com.google.appengine.tools.appstats.AppstatsFilter</filter-class>
    <init-param>
      <param-name>maxLinesOfStackTrace</param-name>
      <param-value>16</param-value>
    </init-param>
</filter>