Google App Engine Datastore does not run live, does locally

Question

I have deployed a Java project to Google App Engine, and the Datastore is not working as it does locally. The project produces a webpage with a summary of various news articles. These articles are taken from the Datastore. The Datastore is updated with new articles every 30 mins via a cron job and Java Servlet. At least this is what happens locally. When deployed live the cron job runs, successfully according to the logs, but there are no entries in the datastore. Has anybody come across a similar problem and have a solution?

Code

@SuppressWarnings("serial")
public class CronServlet extends HttpServlet {
    private static final Logger log = Logger.getLogger(CronServlet.class.getName());

    public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
        try {
            ArrayList<String> feeds = new ArrayList<String>();

            feeds.add("http://feeds.reuters.com/reuters/technologyNews");
            feeds.add("http://feeds.reuters.com/reuters/companyNews");

            RunTagger tagger = new RunTagger();
            tagger.loadRssStreams(feeds.toArray(new String[feeds.size()]));

            ArrayList<Story> articles = tagger.tagArticles();
            uploadArticles(articles);

            log.info("Succesfully updated database");

        } catch (Exception e) {
            log.error("Failed to update database.");
            e.printStackTrace();
        }
    }

    private void uploadArticles(ArrayList<Story> articles) {
        DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();

        for(Story article : articles){
            String url = article.getUrl();
            String headline = article.getHeadline();
            String summary = article.getSummary();
            String tags = article.getTags().toString();
            String feed = article.getFeed();
            String image = article.getImage();

            DateFormat format = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss z", Locale.ENGLISH);
            Date date;

            try {
                date = format.parse(article.getDate());
            } catch (ParseException e) {
                date = new Date();
                e.printStackTrace();
            }

            Key k = KeyFactory.createKey("Articles", url+feed);

            Entity entity = new Entity("Article", k);

            entity.setProperty("url", url);
            entity.setProperty("headline", headline);
            entity.setProperty("summary", summary);
            entity.setProperty("tags", tags);
            entity.setProperty("feed", feed);
            entity.setProperty("image", image);
            entity.setProperty("date", date);

            datastore.put(entity);
        }
    }
}

Cron Job

<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
    <cron>
        <url>/cron/mycronjob</url>
        <description>Update DB</description>
        <schedule>every 1 hours</schedule>
    </cron>
</cronentries>

Log File

Answer 1

I don't see Successfully updated database log entry, which probably means that this servlet never executed - otherwise you would have seen some error message. Make sure it is correctly mapped in web.xml to this URL.

Answer 2

Just a guess: if you cleaned your project prior to uploading it, you may have clobbered the DataStore index configuration info, as described here .

If this is the case, running your devserver locally and exercising all of the relevant DataStore operations, then uploading the app again (without cleaning first) should fix your problem.

Answer 3

1)Scheduled tasks can access admin-only URLs as.

<security-constraint>
    <web-resource-collection>
        <web-resource-name>cron</web-resource-name>
        <url-pattern>/cron</url-pattern>
    </web-resource-collection>
    <auth-constraint>
        <role-name>admin</role-name>
    </auth-constraint>
</security-constraint>