Using Python to send twitter data directly to Google Cloud data storage

Question

How might one send data from Twitter directly to Google Cloud data storage. Would like to skip the step of first downloading it down to my local machine and then uploading it up to the cloud. It would run once. Not looking for full code, but any pointers or tutorials that someone might have learned from. Using python to interact with google-cloud and storage.

Any help would be appreciated.

Answer 1

Here's a blog post which describes the following architecture:

• Run a Python script on Compute Engine
• Moving your data to BigQuery for storage

Here's another one that describes a somewhat more complex architecture, including the ability to analyze tweets:

• Use Google Cloud Dataflow templates
• Launch Dataflow pipelines from a Google App Engine (GAE) app
• In order to support MapReduce jobs