Using Google App Engine to update files on Google Compute Engine

Question

I am working on a project that involves using an Google App Engine (GAE) server to control one or more Google Compute Engine (GCE) instances. The rest of the project is working well, but I am having a problem with one specific aspect: file management. I want my GAE to edit a file on my GCE instance, and after days of research I have come up blank on how to do that.

The most straightforward example of this is:

Step 1) User enters text into a GAE form.

Step 2) User clicks a button to indicate they would like to "submit" the text to GCE

Step 3) GAE replaces the contents of a particular (hard-coded path) text file on the GCE with the user's new content.

Step 4) (bonus step) GCE notices that the file has changed (either by detecting a change or by way of GAE alerting it when the new content is pushed) and runs a script to process the new file.

I understand that this is easy to do using SCP or other terminal commands. I have already done that, and that works fine. What I need is a way for GAE to send that content directly, without my intervention.

I have full access to all instances of GAE and GCE involved in this project, and can set up whatever code is needed on either of the platforms.

Thank you in advance for your help!

Answer 1

You could use the Google Cloud Storage Client to load the files into a Cloud Storage bucket, and have the GCE instances copy files out of GCS and onto the local filesystem. On the GCE side, you could use Object Change Notification to achieve your bonus step 4.

Answer 2

The most straightforward approach seems to be:

1. A user submit a form on App Engine instance.
2. App Engine instance makes a POST call to a handler on GCE instance with the new data.
3. GCE instance updates its own file and processes it.

Answer 3

You can set-up a cron in GAE

https://developers.google.com/appengine/docs/python/config/cron?hl=es

and create a SCP job directly in your code.

How to scp in python?

i hope this work for you.

Answer 4

You can set an action URL in your form to point to the GCE instance (it can be load-balanced if you have more than one). Then all data will be uploaded directly to the GCE instance, and you don't have to worry about transferring data from your App Engine instance to GCE instance.