How can I save some rules or config in google cloud and then call it to process a pubsub topic data in cloud functions?

Question

I am working with google cloud functions. I am coming from a serverbased applications so I don't know much about the serverless world.

I have a scenario where i have two kinds in google datastore ie iot_kind and config_kind. One kind has config which is like a lookup table and other kind has normal payload values.

I am publishing the data from a IoT device into the pubsub topic and getting that data into the cloud functions.

eg IoT payload is

{"id":"213213", 'price': 20, 'name':"some_name"}

then I am querying the config KIND and extracting some data from it like price multiplier

select * 
from config_kind 
where id = "213213" #lets say output is 2

Then I am multiplying the price*2 and saving it to IoT_kind eg value=40

Now in this way I have to make a read every time I get a data into the cloud function which is i think quite expensive and also a bit processing.

Is there any better way of doing it? Like I save latest config somewhere and then whenever payload comes it sees the config and do the processing according to the config and then save it to a database?

And can I do something like make a cloud function and store the latest value there and then call this cloud function from another cloud function to get the values out of it? would it be less expensive or expensive ? Thanks a lot!

Answer 1

Cloud Function is a serverless and stateless service. When an instance is created, it lived for a while and it has been stopped. No disk are mount, all work in memory. That means that you can't save any value in it persistently.

Anyway, you have 3 solutions to reduce the number of config read into datastore

• Even if you can't store persistently data, you can still store them in memory (up to 2Gb of memory, your application footpring included). Thereby, you have to perform the first request to Datastore an then, you can store the config data into a map defined as global variable. Because Cloud Functions can handle only 1 request in the same time you don't have race condition. (It's not the same thing with Cloud Run for example)
• If your config doesn't change often, you can store it into a file and store into Cloud Storage. At the beginning of the function you load the file (and store it into a global variable as before, for reading it only once), and you perform you search into the file. That's work only if the config file size is bellow the function memory limit.
• If the read cost is above 50$ per month, you can use Cloud SQL database instead of Datastore.

Is that make sense for you?