Running a python script on Google Cloud Compute Engine

Question

For a machine learning task at school I wrote my own MLP network. The data set is quite big, and training takes forever. I was alerted to the option of running my script on the Google Cloud Compute Engine. I tried to set this up, but did not succeed (yet).

The steps I undertook where:

1. Create an account
2. Create a VM
3. Open the VM via the browser

Can anyone help me with importing and running my python script into the Google Cloud. Or does anyone have clear a tutorial on how to solve this? I tried finding these myself, but had no success so far.

Answer 1

I finally figured this out so I'll post the same answer on my own post that worked for me here. Using Debian Stretch on my VM. I'm assuming you already uploaded your file(s) to the VM and that you are in the same directory of your script.

1. Make your script an executable

   chmod +x myscript.py

2. Run the nohup command to execute the script in the background. The & option ensures that the process stays alive after exiting. I've added the shebang line to my python script so there's no need to call python here

    nohup /path/to/script/myscript.py &

3. Logout from the shell if you want

   logout

Done! Now your script is up and running. You can login back and make sure that your process is still alive by checking the output of this command:

ps -e | grep myscript.py

If anything went wrong, you can check out the nohup.out file to see the output of your script:

cat nohup.out

Answer 2

There is even a simpler approach to to run code in the background in gcp and in every linux terminal: using screen linux

Create a new background terminal window:

screen -S WRITE_A_NAME_OF_YOUR_CHOIC_HERE

now you are in a background window in the terminal. Run your code:

python3 mycode.py

Exit screen with the hotkeys and the job will keep running on the background.

ctrl + A + D

You can close all windows now. If you wanna go back and see what's happening. Log again in your terminal. And tap the following.

screen -ls 

This one will give you the list of the created "windows". Now find yours and tap

screen -r WRITE_NAME_OF_YOUR_WINDOW

And there you have it :D You can find more commands here

Answer 3

You can use Google Cloud Platform tutorials itself and is very simple to follow. Links are given below

Setting up Python

https://cloud.google.com/python/setup

Getting started

https://cloud.google.com/python/getting-started/hello-world

Please note that you don't have any free tier to run Python 3.x, Standard environment with free tier only supports Python 2.x.

Edit: As per the latest update Python 3.x is default in standard environment

Answer 4

只需导航到放置脚本的目录即可。

python thenameofscript.py

Answer 5

I used Way Script which is great and have a free plan to run every hour

You can check this video to see explanation